TOM survey

(go back)

Introduction

This section of qualitative data analysis is based on three separate yet complementary elements: a survey of TOM users; interviews with authors of TOM ideas; interviews with public officials charged with responding to TOM-generated ideas. These three elements enable us to complete the assessment of how well TOM functioned as a means for enabling e-participation.

The Survey

An online survey was conducted among the registered users of TOM between 30 April and 14 May 2007. The questionnaire was sent to 80 persons who had presented ideas via TOM in the period June 2001 – March 2007. The sample consisted of the authors of the ideas with the highest number of votes cast. The survey did not include commentators, voters, and passive users of TOM (portal followers).

The survey contains the answers received from 25 respondents (a 31% completion rate).*

The survey aims to analyse the “lifestory” of an idea presented via TOM, focusing on how and why the idea came into being, the efficiency and user-friendliness of the commenting, editing, and voting phase of the portal as well as the eventual outcome. The survey included the following questions:

1) What did you know about TOM before presenting your idea?

Keywords: How did you come across TOM? Had you previously taken part in commenting and voting before presenting an idea? Did you have an overview of how TOM works before presenting the idea etc?

2) How and why did your idea come into being and how did it get to TOM?

Keywords: Media influence, relevant and “hot” topic in current affairs, personal problems. Was the idea that of one person or was it a group initiative? What was the preparation of the idea like? What happened outside TOM (in the Internet, discussions among friends etc)

3) What was your experience of using TOM like (disregarding the quality of comments, voting result, and the eventual government response)?

Keywords: Positive examples of using TOM. What did you learn and what should one know before presenting the idea? Did TOM help you in formulating, presenting, and processing the idea?

4) Could you evaluate the comments on your idea, the voting process, and the answer received from the government?

Keywords: Was there any discussion on the presented idea? Did you notice a clear “opposition” to the proposal and whether the comments influenced the voting outcome? What was the result of the presented idea and its possible impact? What was the answer from the government department like?

5) What functions should be added to TOM in order to make it more convenient to use and more effective? What should be changed?

6) What could TOM be used for (in addition to its present function)? What would you do with TOM given the possibility?

Survey results and main findings

1) What did you know about TOM before presenting your idea?

All the respondents were already familiar with the existence of TOM before presenting their idea. The main sources of information included: Estonian Government websites, the Estonian Parliament, internet portals, online newspapers, TV, newspapers. None of the respondents had used TOM before (either for commenting or voting).

Most of the respondents considered themselves not to be active users of TOM, either before and after the idea was presented. Furthermore, a decrease in visiting the portal was mentioned several times.

However, the existence of the portal was highly regarded and the information presented on the portal explaining how to use the tool and propose ideas was seen as clear and sufficient. Nevertheless, in certain cases technical problems of the portal (inability to register, etc) were mentioned.

Problems: Most respondents heard about TOM in the immediate aftermath of the portal’s launch in June 2001. After the initial burst of publicity and attention, interest in the portal as well as media coverage decreased considerably and the possibilities of finding information related to the TOM or promoting its existence is rather scarce. For example, in 2001-2002 there was a link to TOM from Estonia’s biggest Internet portal (www.delfi.ee) that was simply removed after 2002. Many of the respondents could not recall the last time they had heard anything about TOM. A very low media coverage is directly related to the low number of new registered users of the TOM as well as active use of the portal (low number of voters, comments, etc).

Possible solutions proposed: To add a TOM link to all the websites of government institutions as well as online newspapers and large national portals, e.g., www.delfi.ee. Also, articles in newspapers, magazines, etc should be followed by a comment like “the issue could be discussed also in TOM” or for further debate visit TOM”, adding the link to the latter.

2) How and why did your idea come into being and how did it get to TOM?

A vast majority of respondents presented the idea due to personal reasons (either work- or family-related). Also, media interest in the particular issue was mentioned several times (e.g., the subject matter was touched upon in newspapers or other articles). In some cases, respondents expressed a need to draw other citizens’ attention both to certain problems and possible solutions alongside a desire to help governmental institutions in their work, legislation preparation in particular.

All the respondents presented the idea alone (not with a group or co-presenter) and most of them had not used any external assistance when formulating the idea. Only in one case was the idea generated in a brainstorming session that was followed by drafting the idea together with an expert group.

Respondents agreed that they had not spent enough time formulating the idea (according to 2 answerers, it took them an hour) nor in advertising the proposal or making active use of others (for commenting and voting). However, in three cases the idea had been under discussion in online newspapers or other internet portals and in one case, the idea presenter had been contacted by a TV channel for an interview. The latter due to the author’s activism and personal contacts, not the help of TOM managers or public officials.

Problems: The ideas are presented by a single person and are not usually formulated without requisite attention and the use of external expertise. In addition, idea presenters are neither active in engaging others in the preparation, commenting, and voting phase of the proposal nor in drawing public attention to the problem through other channels (newspapers, the Internet, friends, coworkers, etc). This serves as a strong reason explaining the low quality of the presented ideas as well as the passive attitude of other citizens in the idea-generating process.

Possible solutions proposed: To support the activism of the idea author in promoting the proposal as well as taking part in the commenting phase and drawing attention to the problem in other fora (TV, newspapers, etc). To draw the author’s attention to possible ways of getting expertise and assistance in both formulating and editing the idea.

3) What was your experience of using TOM like (disregarding the quality of comments, voting result, and the eventual government response)?

From the survey data gathered it can be concluded that the general attitude to TOM is positive as well as the experience of presenting and processing ideas. The structure is seen as logical and the opportunity to follow the procedure after presenting the idea adequate.

The instructions on how to use TOM for various purposes (voting, commenting, editing) is considered easy and no major technical changes were proposed. However, it was suggested to add ways of receiving information on TOM and the new ideas presented. For example, tag-based (keyword-based) e-mail subscriptions.

The inability to edit a proposal when in the commenting phase was considered problematic in several cases (both by commentators and the author of the idea).

Also, the issue of verifying the person’s identity with a national ID card was raised a couple of times (whether or not the ID card should be used for user registration, authoring an idea, commenting, and voting). Note, however, that in 2005 the Estonian State Chancellery changed the rules of TOM and introduced a requirement of using an ID card when voting (only the votes given by the persons verifying their identity with ID card were counted). As this change was highly criticised, the State Chancellery decided to abolish the requirement and, currently, registration with ID card is optional and there are no restrictions as to commenting, voting, or authoring an idea. However, this subject is a constant matter of debate both amongst TOM users and public officials.

It can be summarised that the problems raised are related to content and impact rather than technical. Still, the visual design of the portal was criticized and considered old fashioned.

4) Could you evaluate the comments on your idea, the voting process, and the answer received from the government?

Problems: The majority of respondents identified a lack of user discussion as one of the main drawbacks of the portal. The number of comments had not met authors’ expectations and the disengagement of public officials was considered negative.

However, the quality of posted comments are highly regarded and considered valuable and interesting, even if the voting result was negative. The reputation of TOM is considered high and the latter is seen as the reason for not posting low quality or nasty comments that are the norm in many Internet forums.

Voting results and the actual voting procedure were neither considered problematic nor an obstacle for using the portal for presenting new ideas in the future.

In addition to the lack of discussion, public officials’ answers to the TOM-generated ideas are highly criticised. All the respondents have received negative answers (i.e., the presented idea is not to be implemented) and all the answers are described as being too general and mealymouthed regarding the actual decision. This is interpreted by respondents as the sign of an unwillingness on the part of civil servants to contribute to the possible implementation of an idea, which respondents believe is merely seen as extra work by these public officials. That contributes to the respondents pessimism regarding the usefulness of the portal that can be illustrated with the statements like „nothing will change anyway”, „our opinion doesn’t count” etc.

Still, some respondents point to the the civil servants’ inability to implement the idea due to their low status and lack of higher level political support as part of the reason an idea was dismissed.

The lack of relevant knowledge and information necessary in order to be able to post comments and participate in thorough discussions is mentioned upon several times by survey respondents. As most of the TOM-generated ideas concern a very specific policy area and require certain background information, the number of people commenting on the presented ideas is low and the persons involved in commenting and voting tend to be the same ones.

Voting results and voting procedure is not seen as problematic or an obstacle for using the portal for presenting new ideas in the future.

Possible solutions proposed: To provide background information on the TOM-generated idea and the subject matter as well as explanations regarding terminology used; to categorise the presented ideas according to subject matter so that it would be possible to get information on the related ideas discussed in the portal beforehand; to involve experts, lawyers, and public officials to assist the idea author and contribute comments on the issue; to hire a professional moderator that should “lead” the conversation between portal users, idea presenters, and public officials; in addition to public officials in the relevant ministries, the ideas should be forwarded straight to higher government institutions (e.g., Parliament) or relevant expert committees (e.g., in the Parliament).

5) What functions should be added to TOM in order to make it more convenient to use and more effective? What should be changed?

Technical changes for making the portal more functional and easier to use were not proposed by any of the respondents. Overall, respondents focused instead on the following issues:

– anonymity: introducing the requirement of identifying TOM users with national ID card;

– pseudonyms (whether or not to allow users to participate without their real name);

– public awareness (using other channels like the Internet portals, online newspapers, TV, radio, and newspapers for promoting TOM);

– involvement of experts, moderators, opinion leaders in commenting the presented ideas and leading the discussion between TOM users and public officials;

6) What could TOM be used for (in addition to its present function)? What would you do with TOM given the possibility?

The majority of respondents wish to see public awareness of the portal raised as the number of TOM users is seen as the key element in the portal’s usefulness and continuation.

Many respondents would like to foster ways of engaging TOM in wider discussions in society (links to other Internet sites, news in TV, newspapers, radio etc, links to other relevant documents).

The possibilities of engaging opinion leaders, experts, higher officials and politicians in discussion and commenting on the TOM proposals are mentioned in several cases.

It is considered vital to integrate TOM with other services: a public law database, Draft Acts and statute amendments etc.

Summary of the interview with idea authors

In addition to the web survey analysed above, a smaller number of authors of ideas with a high vote count or the possibility of implementation were contacted for private discussion that allowed us to ask more questions and test the viability of solutions we have considered as possible solutions to problems visible in the technical website and database analysis.

The main problems of TOM are perceived to be:

  • inactivity and lack of publicity
  • discussion was more active when TOM ideas were co-published by major Estonian portal delfi.ee – this brought new users and added a more familiar discussion platform
  • too many steps during the process of idea makes it difficult to follow
  • no feedback to users about the progress of idea
  • discussions tend to get personal and move away from topic
  • anonymity has not been a problem, as it allows also participation of citizens who are working as civil servants (there are reportedly cases with public petitions where signers have felt pressure from official channels afterwards)
  • TOM should be integrated with other services, like a public law database, database of law amendments etc – perhaps making amendments commentable?
  • no visible influence of TOM ideas beyond TOM: perhaps ideas should be circulated to respective committees in parliament or coalition council
  • lack of structure: need for a categorization of ideas, possibly also relations to law amendments in process (ideas tend to reflect the wider discussions in society)
  • there is a need to link outside documents and discussions to TOM-generated idea

Conclusion: simplify the process, keep anonymity, make sure users return to participate in subsequent phases, find a way to create a buzz beyond TOM and engage citizens in wider discussions by linking to other sites and external documents, provide structure.

Dialogue or monologue?

The lack of dialogue between the two participating sides – citizens as idea authors and government respondents – was especially highlighted during discussion.

  • active subjects create dialogue, but a lot of subjects tend to get dominated by skeptics as there is no visible path to a policy solution
  • TOM should use solutions like email notifications to help users return to the discussion
  • how to give more weight to ideas, get less formal answers: there exists the possibility to invite related NGOs to file support after voting, that could also help to find partners who could lobby for the idea
  • there is a need for discussion after receiving a government response: currently there are very few comments on answers since users are not notified about them; a possibility of re-submitting an edited version of idea following a negative answer is needed
  • raise the bar for voting: quorum of votes required, support must be larger than a simple majority
  • including working plans of ministries would make it easier to select subjects government ministries might be willing to discuss
  • statistical feedback is necessary for TOM users: how many ideas have been sent to which ministry, what has been the response (negative, positive, has the idea been implemented)
  • “send to a friend” – some simple campaign solutions that let users widen the circle of participants
  • monitoring functionality – users should have multiple options (email, RSS etc) for receiving information about changes to their idea or ideas on their field of interest
  • how to find out during creation of the idea whether there has been any earlier ideas on the same topic, possible providing input? if the ideas were tagged/categorized users could be asked to provide similar information and TOM could provide easy access to ideas: reading, linking as reference etc

Conlusion: activate users and provide simple “campaign tools”, create ties with previous ideas, bring users back to discuss government responses and possibly continue the process with a new and improved idea, during the creation of new ideas find a means to inform users about possibly-related ideas that could have solved the same problem or might prove valuable reference.

Summary of the interview with representatives of ministries

To understand the possibilities of and problems with TOM a meeting with representatives of relevant ministries was organized. Participants were public servants who deal with the questions the State Chancellery has directed to particular ministries in order to have an overview of the process – actual answers are prepared by specialists of the field and approved at secretary general or deputy secretary general level.

The first problem that has influenced the impression of TOM amongst public officials is the fact that ideas that have passed the system with very few votes – answering the ideas is a burden on specialists. Despite this and the fact that there tend to be „loyal correspondents“ who create a sizeable number of ideas the quality was perceived to be higher than that of the general ministry inbox, so in case the system is adjusted to produce a higher votecount it might actually lessen the burden of answering letters from citizens.

Contrasting an Information request vs TOM idea

The Estonian Public Information Act grants citizens wide rights to request information and a large part of communication with ministries falls under PIA: they are answered using available information within 5 days. TOM ideas are considered different, as they arrive with the resolution of State Secretary and need typically a more in-depth answer that takes of course more time to deliver but is also put through a full approval round and signed by a rather high official meaning this is considered an “official governmental position”. Unfortunately this also seriously limits the possibilities of dialogue: having a position means defense, not discussion. There is no way to engage government officials in discussing the answer publicly on TOM and most follow-up ideas will be considered as an attack on this official position and automatically barred.

Part of the problem is that “politicians decide, public servants execute” – meaning that original ideas directed to the ministry level will be politely rejected unless they fall into a subject category that is already in the pipeline or are effortlessly implementable corrections of minor mistakes.

Sidenote: the same problem was suggested by idea authors with a possible solution of circulating the ideas to parliamentary commissions or a coalition council. We presented this as a possible solution to the public officials and it was rejected.

How to allow discussions? If the approval level is brought down it might create a chance to implement more TOM-generated ideas, but even then the answers-comments in the follow-up discussion should go through an approval round in ministry, as even personal opinions of public servants tend to get interpreted as an official position. Which brings us again to the issue of anonymity or pseudonymity – it should be possible to participate without making one’s real name known. But how to manage such a system and avoid the current situation where it is dominated by few users?

Good ideas will be picked up and used, in case they are remembered at the suitable moment. Could there be a solution to send a reminder of positively-answered ideas once there is a related statute amendment in process? Adding subject categorizations to ideas is considered helpful, as this would facilitate finding ideas later. Also adding information about the status of the idea – implementable, rejected etc – was considered useful.

What kind of meta-information might be added to answer? Categorization of the answer, including pointers to laws that it might influence so that the system can notify users about ideas when a law is to be changed in future (sidenote: part of the development of participation tools at State Chancellery is also tied to the development of various databases).

Idea authors suggested that after the idea has been voted upon interest groups should be able to add their letters of support – this is considered a good option by public officials.

Conclusion: the current setup does not facilitate discussion, but we cannot point to any good alternative; the public officials’ attitude towards TOM could be improved if the system had more users (more authors, more people voting for ideas) and were it to generate ideas that are of a higher quality than the average ministry inbox content; acceptance of ideas would be higher, if they are related to the actual working plans of the ministry; implementation of ideas could be higher if there were a solution to remind civil servants about them once a related law is in process. However, this is not a exhaustive list of how TOM can be made more useful to ministries.

** The surveys were conducted in the Estonian language. The resulting “raw material” is not available in a translated version.