Georgian public increasingly unaware of what the European Union Monitoring Mission does

[Note: This article originally appeared at OC-Media. The article was written by Dustin Gilbreath, a Policy Analyst at CRRC-Georgia. The views presented in this article are the author’s alone and do not necessarily reflect the views of CRRC-Georgia or Europe Foundation.]

As much as 81% of the population of Georgia doesn’t know what the European Union Monitoring Mission (EUMM) does, according to the 2017 Knowledge of and Attitudes towards the European Union in Georgia survey funded by Europe Foundation and implemented by CRRC-Georgia. This lack of knowledge has increased over time, as has the prevalence of incorrect information about the EUMM’s mission. This represents a missed opportunity for the EU’s communications in Georgia.

On the survey, respondents were asked, “What does the European Union Monitoring Mission do in Georgia?” A large plurality of the population (41%) reported they did not know what the EUMM does. The second most common response (25%) was “supports the implementation of democratic and market oriented reforms,” an incorrect answer. The third most common response (19%) was “supports the stabilization of the situation in the areas affected by the August 2008 war,” the EUMM’s actual mission.

The share of the public aware of what the EUMM does has declined over time. While in 2009, 39% of the population knew what the EUMM did, 19% did in 2017. This decline may stem from the relatively high salience of the Monitoring Mission in the years immediately following the 2008 August War with Russia, although no data exists which would confirm this.

Besides the decline in knowledge of what the EUMM does, inaccurate information about the organization has become more prevalent. While only 24% of the public gave an inaccurate answer to the question in 2009, 38% did in 2017. Notably, there was a large increase in don’t know responses in 2015 and a sizable decline in 2017. Rather than an increase in correct responses in 2017, however, the data suggests that a lack of knowledge was replaced by incorrect information.



The lack of knowledge about the EUMM is most pronounced in ethnic minority settlements, with only 2% of individuals in minority communities correctly responding to the question. The lack of knowledge in minority settlements should come as no surprise given that surveys in these communities regularly have high rates of don’t know responses.

In contrast, those living in rural settlements with a predominantly ethnic Georgian population provide the correct response most often. The fact that the rural population is more informed than urban populations may stem from the EUMM’s rural presence. While the organization has offices in four urban settlements – Tbilisi, Mtskheta, Zugdidi, and Gori – they regularly patrol the rural areas surrounding the administrative boundary lines with Abkhazia and South Ossetia.

Although the public is increasingly unaware of what the EUMM does, about 2/3 of people who provided incorrect answers to the question about the EUMM’s mission would like to have more information about the EU. About a quarter even want information specifically about the EU’s role in resolving Georgia’s territorial conflicts.

The lack of knowledge of what the European Union Monitoring Mission does in Georgia may represent a missed opportunity for the EU. While no data is available about the attitudes of people who have had contact with the EUMM, previous research in Georgia has suggested that individuals contacted by NGOs report greater trust in them. The EUMM, given its public service mission, may receive a comparable boost from contact with the public. Hence, the EU should consider increasing its outreach and communications related to the EUMM.

Survey incentives: When offering nothing is better than offering something

Why do people take the time to respond to surveys in Georgia? A telephone survey experiment CRRC-Georgia carried out in May 2017 suggests that small financial incentives may actually discourage people from participating in surveys. This finding suggests people may respond to surveys for intrinsic (e.g. because they are curious or want to help) rather than extrinsic reasons (e.g. doing something for the money).

During the experiment, 2320 respondents were asked whether they would be willing to participate in future phone surveys. Half were offered a GEL 2 transfer (about US$0.82 at the time of the survey) to their telephone in exchange for doing so. The other half was not offered any incentive. Respondents were randomly assigned whether they were offered an incentive or not.

The difference in responses isn’t impressive, but is statistically significant. Approximately 4% fewer people said they would participate in future surveys when offered the incentive. The chart below displays the effects of different variables on willingness to participate in future surveys in terms of odds ratios. The first model looks at the effect of being offered GEL 2 alone, while the second model controls for age, sex, and settlement type. The odds of someone responding positively when offered the incentive where approximately 0.8 to 1 in both regressions, meaning that the offer made it less likely that individuals would be willing to participate in a future survey. The second model also shows that except for those in the 36-55 age group, demographic characteristics have no effect on people’s likelihood of agreeing to participate in future surveys.


Note: For the regression models presented above, “No” was coded as the base category, “Don’t know” as the second category, and “Yes” as the third category. The logic behind coding “Don’t know” as a middle category is that the person is not refusing to participate in future surveys, but rather is saying they might or might not. 

The results of this experiment show that people in Georgia are slightly less likely to want to participate in future surveys if you offer them a small amount of money compared with offering nothing. This finding may at first seem strange. However, a potential explanation is that GEL 2 may have seemed like a paltry sum, and people may have been offended. In turn, rather than engaging people’s intrinsic motivations like a sense of duty to help others or simple curiosity, the offer of GEL 2 could have activated people’s extrinsic motivations. In turn, the extrinsic incentive wasn’t large enough to counter the loss in the intrinsic value of participation, at least on average.

For a look at a comparable effect in a very different context, this study on attitudes towards having nuclear waste facilities found a similar pattern: when offered money, people were less likely to support having such a facility in their community.

Have other thoughts on what might have encouraged those offered GEL 2 to participate in future surveys to decline participation more frequently? Let’s have a conversation on Facebook or Twitter.

Will an Independent Mayoral Candidate Bring Political Change to Georgia?

[Note: This piece was originally published at New Eastern Europe. It was written by David Sichinava. David is a Senior Policy Analyst at CRRC-Georgia and Assistant Professor at Tbilisi State University. The views expressed in this article do not necessarily reflect the views of CRRC-Georgia, the National Democratic Institute, Tbilisi State University, or any other affiliated entity.]

A June 2017 survey CRRC-Georgia carried out for the National Democratic Institute suggests that Aleksandre (Aleko) Elisashvili, an independent candidate and prominent grassroots activist could win the Tbilisi Mayoral elections, if the elections enter  a second round, which the polling also suggests is a possibility. While only election day will tell the ultimate result, if Elisashvili does win, it could be the start of a shake up of Georgian political life.

Local elections are set for October including the election of mayors in several self-governing cities. In Georgian local elections, the Tbilisi Mayoral race is considered the main race, given that the city contains roughly one third of the country’s population. For the ruling Georgian Dream party, former Energy Minister and Vice Premier Kakha Kaladze is running for the post. The United National Movement, which following defections consists of loyalists to Mikheil Saakashvili, has nominated Zaal Udumashvili, a former anchor on Rustavi 2, the country’s largest TV station. European Georgia, a party consisting of defectors from the United National Movement has nominated Elene Khostaria, a former official in the UNM government and currently an MP, for the mayoral race. The race also features Irma Inashvili, the leader of right-wing Alliance of Patriots, and Giorgi Gugava, endorsed by the Labour Party of Georgia.

Besides the party-affiliated candidates, Aleko Elisashvili, an independent who rose to prominence through his engagement with a variety of grassroots movements aimed at preserving Tbilisi’s urban heritage and currently serving on the Tbilisi city council, has thrown his hat into the race.

June 2017 polling CRRC carried out for NDI places Kaladze in the lead with 37% of the vote among likely voters. Elisashvili came in second with 22%, followed by Udumashvili (16%) and Khoshtaria (5%). According to the survey, 16% of likely voters in Tbilisi are undecided or refused to answer who they would vote for.

If undecided voters are distributed equally between parties, a simple but often accurate way of forecasting election outcomes in the absence of a large number of surveys, it suggests that Kaladze would garner about 42% of the vote in the first round of elections. If Kaladze receives less than 50% of the vote, the electoral threshold to win outright in the first round of Georgia’s mayoral elections, it would lead to a runoff. Given the relatively small sample of likely voters in Tbilisi in the survey and the fact that elections were still at least two months away, it is still unclear whether there will be a run-off, though it is a possibility.

Even though Elisashvili is polling 15 points behind Kaladze, in a runoff, Elisashvili could win the race. In a TV interview, both Udumashvili and Khoshtaria expressed their willingness to support any pro-Western opposition candidate against Kaladze in a second round. Assuming those parties supporters would turnout for Elisashvili, the polls suggest he would garner about 54% of the vote. Although Udumashvili is neck-and-neck for the second place, in case of runoff, he does not have Elisashvili’s declared support. Again, keeping in mind the relatively small Tbilisi sample and the level of error associated with it, this places Kaladze and Elisashvili in a neck-and-neck race.

That means that the campaign, and the race for undecided voters matters. While Kaladze has vowed to replace Soviet block flats with modern housing and promises to further invigorate Tbilisi’s lively nightlife, Elisashvili’s campaign has focused on urban politics and policy. He has attacked Tbilisi mayor’s office for corruption and is positioning himself as an anti-establishment candidate. Elisashvili’s negative stances towards large real estate developers and Tbilisi’s investor-induced construction frenzy indeed fit the anti-establishment image he’s cultivating.

A closer look at the NDI survey suggests Elisashvili is doing better than Kaladze among those voters who do not identify themselves with any party, almost 40% of the city’s population. Surprisingly for a grassroots activist, Elisashvili’s supporters are more likely to be older and well-off - two demographic groups who generally turn out to vote. They are also more likely to think that there is corruption, nepotism and a lack of professionalism in Tbilisi mayor’s office. Finally, voters who think that environmental pollution is the most important public goods issue in Tbilisi - which has consistently been viewed as the most important public goods issue in the capital - are also leaning towards Elisashvili, as are those who find recent construction in residential neighborhoods unfit to the area.

These political leanings in combination with support for Elisashvili make sense given his background. He was instrumental in founding Tpilisis Hamkari, an activist group concerned with preservation issues. Since its inception in 2005, Hamkari has led protest rallies to save several landmark buildings across Georgia’s capital from demolition. Started by a handful of activists, the movement climaxed in 2011 and 2012 when it drew hundreds to rallies attempting to save Gudiashvili square, a small pocket of urban green area in old Tbilisi. Apart from his work as an activist, Elisashvili also hosted political talk shows on the opposition-associated Kavkasia TV and served as head of the state parole board.

Elisashvili’s activism has contributed to his past political successes. In the 2014 local elections, he ran as an independent, winning a seat on Tbilisi city council in a narrowly fought race with candidates endorsed by the Georgian Dream and the United National Movement in the affluent Saburtalo district. At the city council, he fiercely criticized legislation the Georgian Dream-dominated city government proposed. Importantly, Elisashvili opposed the Panorama Tbilisi construction project which was endorsed by Bidzina Ivanishvili, Georgia’s former prime minister and the wealthiest man in the country.

Whether or not Elisashvili manages to maintain or increase his support, many care about the causes the independent candidate stands for. Georgians are extremely skeptical towards local government and issues like environmental pollution and urban development have become increasingly salient. Political scientists still argue whether such issues matter for party politics in post-communist societies. However, the polling and movements in different contexts increasingly suggest that activists have the potential to challenge and shake up city politics through focusing on these issues. The recent success of an anti-establishment social activist, Ada Colau, in Barcelona’s mayoral race is yet another example.

While, it is too early to say whether Elisashvili will be able to turn his popularity into a successful bid for the mayor’s office, the fact that an independent candidate with a relatively unusual political platform is polling strongly could suggest the winds of change are afloat on the political scene in Georgia. Whether those winds will lead towards a revival of Georgia’s democracy, or whether the 2017 local elections will be another round of hope followed by disappointment at the lack of the opposition’s political acumen like in the 2010 local elections awaits election day.

The data used in this article is available here. The input output model used to distribute voters is available here.

Visa liberalization: How much do people in Georgia know about the conditions of visa-free travel to the EU?

CRRC’s previous blog posts have shown that the population of Georgia had rather moderate expectations of the recent visa liberalization with the Schengen zone countries, especially when it comes to the question of how much ordinary people will benefit from it. Europe Foundation’s latest survey on Knowledge of and Attitudes towards the European Union in Georgia, conducted in May 2017, provides a more nuanced understanding on how people in Georgia feel about this process and to what extent they are familiar with the conditions of visa liberalization.

In May 2017, only about 1% of the population of Georgia reported having not heard of visa liberalization. A majority, 64%, reported being glad to have the possibility to travel to the Schengen zone countries visa free, although only 16% believed they personally would take advantage of the visa-free regime in the next 12 months. About a third of the population said visa liberalization did not matter for them, and a rather small minority (4%) reported not being glad about visa liberalization.

Five major conditions have to be met by Georgian citizens to enjoy visa-free travel: they should be able to provide a return ticket, travel insurance, proof of financial means to cover their trip expenses, the address where they will be staying during the trip (a hotel reservation or the address of people inviting him/her) and hold a biometric passport. The Georgian government has implemented a large-scale information campaign to spread information about the conditions of visa liberalization as widely as possible. In order to learn how effective this campaign was, the survey included an open question, “Which are the documents that a Georgian citizen needs in order to travel to the Schengen zone countries visa-free?”

According to the findings, people best remembered the requirement of having a biometric passport – 78% named this condition of visa-free travel. Much smaller shares remembered the other conditions: 45% named financial means, 40% a return ticket, 34% the address where a traveler will be staying during the trip, and only 24% named travel insurance. Understandably, those planning to travel to the Schengen zone in the next 12 months demonstrated a better knowledge of the conditions of visa-free travel. However, the differences were not impressive, especially taking into consideration the small size of this group and thus a relatively larger margin of error.

Overall, only 12% of the population named all these conditions during the survey. Rather surprisingly, the rural population and those living in urban settlements outside the capital “scored” better in this exercise compared to the population of the capital and ethnic minority settlements. On the other hand, 18% failed to name any of the five conditions of visa-free travel. The population of ethnic minority settlements demonstrated the poorest knowledge.

Importantly, as of May 2017, a quarter of the population of Georgia mistakenly believed that as a result of the visa-free regime, Georgian citizens obtained permission to work in the EU. The share increases to 34% among those who say they will travel to a Schengen zone country in the next 12 months. Thus, a preliminary look at the findings about knowledge of the conditions of visa liberalization for Georgian citizens suggests that the information campaign needs to expand, and become more intense and targeted to potential travelers.

The datasets and findings of all waves of Europe Foundation’s survey on Knowledge of and Attitudes towards the European Union in Georgia are available on CRRC’s online data analysis platform. A report focused on the 2017 data is available here.

Prioritizing the personal: People talk more about personal issues than political events

There is nothing new in the idea that, in general, people would primarily be interested in their own lives, rather than in social or political events. In other words, social and political events will, most probably, be overshadowed by events in one’s personal life. CRRC’s 2015 Caucasus Barometer (CB) survey data provides more detailed insights on this. In this blog post, we compare answers to two CB questions: “When you get together with your close relatives and friends, how often do you discuss each other’s private problems?” and “When you get together with your friends and close relatives, how often do you discuss politics / current affairs?” in Armenia and Georgia.

The population of both countries report discussing private problems with much higher frequency than politics and/or current affairs. Interestingly, while the populations of the two countries report rather similar low frequencies of discussing politics and/or current affrairs, the population of Georgia reports discussing private problems frequently almost twice as often as the population of Armenia.


Note: Originally, 10-point scales were used for these questions, with code ‘1’ corresponding to the answer “Never” and code ‘10’ corresponding to the answer “Always”. For the charts in this blog post, the original scales were recoded into 3-point scales, with codes 1, 2 and 3 combined into the category “Rarely”, codes 4 through 7 combined into the category “With average frequency”, and codes 8, 9 and 10 combined into the category “Frequently”. Answer options “Don’t know” and “Refuse to answer” (less than 1% of the cases) were excluded from the analysis. 

When looking only at the most radical answers on the original 10-point scales (“Never” and “Always”, i.e. codes 1 and 10), in both Armenia and Georgia the share of those who report always discussing politics and/or current affairs is much less than the share of those who report never discussing these issues. In Armenia, 8% report always discussing politics and/or current affairs when they get together with close relatives and friends, as opposed to 29% who report never doing so. The respective shares are 11% and 25% in Georgia.

When it comes to the shares of the population recording the most radical answers about the frequency of discussing private problems, the pictures in the two countries are quite different. While practically equal shares report discussing private problems with close relatives and friends in Armenia either always (14%) or never (16%), in Georgia, four times as many report always discussing private problems when they get together with close relatives and friends (29%), compared to 7% who report never doing so.

Even when researchers rely on self-reported information only, as is the case with these CB questions, a high frequency of discussing certain issues reflects people’s interest in them. In Armenia and especially in Georgia, few people spend time talking politics. Not surprisingly, these are mostly older people. While there are no large male-female differences, the reported frequency of discussing politics with close relatives and friends differs for the population of different settlement types. Most surprisingly, the findings in this respect are rather different for the capital cities of Armenia and Georgia.

Thus, although the general patterns of frequency of discussing different issues with close relatives and friends are similar in Armenia and Georgia, there are certain important differences that would merit further research. Specifically, one important question to answer is, are Armenians – especially those living in Yerevan – much more reserved while discussing politics?

CRRC’s Caucasus Barometer and other survey data is available at our Online Data Analysis portal.

Get credible! ... Or a modest proposal to implement pre-registration in think tank research

[Note: This post first appeared at On Think Tanks. It was written by Aaron Erlich Assistant Professor of Political Science and Founding Member of the Centre for Social and Cultural Data Science at McGill University and Dustin Gilbreath a Policy Analyst at CRRC-Georgia. The Caucasus Research Resource Centers in collaboration with Aaron Erlich and Caucasus Survey recently announced a pre-registration competition for articles that will use the 2017 Caucasus Barometer Survey. This post is a reflection on how and why think tanks can and should use pre-registration based on the experience of setting up the competition and a summer workshop on pre-registration hosted at CRRC-Georgia in summer, 2017.]

It’s almost a cliche to say that think tanks operate on the basis of credibility. The media, politicians, and some in the general public have increasingly questioned think tanks’ credibility in recent years, with think tanks and tankers becoming increasingly thought of as lobbyists under a different name. Think tanks are not the only ones experiencing a credibility problem. Social science outside the think tank world is also in the middle of a credibility crisis. This crisis stems from the lack of reproducibility of results, scandals related to data fabrication, reliance on small sample studies, and questionable data analysis practices in the search of statistical significance. In response to this crisis, one proposal that aims to ameliorate the situation is the pre-registration of studies. Pre-registration not only represents an opportunity for social science, but also for think tanks to increase the credibility of their work, lighten workloads, and increase donor independence.

What is pre-registration?
A pre-registered study is one where research design elements like sample size, hypotheses, any experimental protocols, and statistical analyses are defined, justified, and placed in a secure registry prior to actually carrying out data analysis. Usually, this means registering the study prior to data collection. However, in some cases one can pre-register a design while data collection is ongoing or before the data are available to the researcher.

For example, the Caucasus Research Resource Centers (Aaron’s former employer and Dustin’s current) is currently holding a competition for papers on foreign policy preferences in the South Caucasus based on the 2017 Caucasus Barometer survey. The survey, at the time of writing, is entering the field and is expected to be complete at the end of October. However, the data itself will not be released until December. To participate in the competition, researchers must register their research design at the Open Science Foundation registry (one of a number of reputable locations to register a study) and then submit their paper based off their pre-registration (without results) to the journal Caucasus Survey. The papers will be reviewed and accepted or rejected without the results of analyses, hence taking away the incentive to find statistical significance.

Like in other studies, analysis commences once an organization has collected data. However, in contrast to an unregistered study, after data collection a researcher need only focus on carrying out the analysis described in their pre-registration plan (or even simply run pre-written code for analysis) and insert the tables and graphs into their report. The bulk of the registration document serves as the report, hence front-loading the writing. Any exploratory data analysis, not described in the pre-registration, is reported as such in the final report.

Why would a think tank pre-register a study?
From the perspective of a think tank, there are reputational advantages as well as more subtle bonuses for managing a think tanks’ workload. The most important advantage of pre-registration will likely be that it increases the credibility of the think tanks’ findings. While in the past, research consumers often simply considered quantitative work robust, today issues surrounding replicability, and statistical modeling like hacking data for statistical significance at the holy 5% level have cast a large shadow over a great deal of quantitative work. Pre-registration precludes statistical hacking among other issues like researcher degrees of freedom, thus leaving fewer avenues of attack for potential critics.

Besides increasing credibility, pre-registration of research design can be particularly valuable for think tanks who work on commissioned studies. While philanthropy is one of the main sources of funding in the United States, in the developing world, think tanks often survive on service contracts and grants for studies on specific issues. Pre-registration is beneficial for both the academic reasons outlined above as well as for the think tank’s work load and independence.

When it comes to workload, a pre-registered study shifts a great deal of effort to project start up, but has the potential to decrease the ultimate workload. Because hypotheses are specified ahead of time, researchers will have to start writing out their expectations instead of focusing solely on design at the start of projects. This means that clients need to agree beforehand on what analyses the think tank will perform, and that agreement can be put into the deliverables of any contract. In this manner, donors who contract research could be constrained in their ability to request more analyses at the end of the project (at least without paying and specifying that these were not pre-registered results). In turn, this prevents researchers from needing to run (potentially hundreds) of additional analyses at the end of a project, when the client is unsatisfied with the results for whatever reason or curious about some other result they had not thought of ahead of time.

When it comes to independence, in the current environment, many donors do not hesitate to pressure researchers to produce results supportive of donors’ positions. With a pre-registered study, researchers have listed out the exact analysis they will implement beforehand along with their expectations about the results of the analysis. Donors who have have been educated in the way this process works can use pre-registration to make stronger arguments. Moreover, the analysis can be built into the deliverables of the project. Hence, donors will be less facilely able to suggest a different analysis or measure in place of the one the researcher chose at the start, thus decreasing the number of avenues through which donors can apply pressure, particularly since changing the analysis would have cost implications.

Many in the think tank community might be skeptical of this proposal. Afterall, donors hold the purse strings. Still, we think that the process will benefit many donors. In many cases, donors want the highest quality study possible, even when motivated by short term goals, because it will help them advance their agenda. While if the donor is so baldly seeking a single result and not willing to accept anything else, we accept the fact that the proposal won’t work. In many case, however, we think donors would take enhanced quality at no cost in exchange for a loss of some control over the final product.

Pre-registration does have limitations and drawbacks. For example, it only works when a researcher can credibly demonstrate that they do not have access to data before pre-registering. Moreover, although it is likely the model could be applied to qualitative research in some form, to date, the model has yet to be implemented widely. Since qualitative research arguably comprises the majority of think tank research, the scope of use is somewhat limited.

While pre-registration is not panacea to the problems of social science or the problems think tanks face, it is a tool for think tanks to consider, which can enhance credibility and potentially decrease workloads and increase independence - three things we don’t think many tankers would be against.

Dustin Gilbreath is a Policy Analyst at CRRC-Georgia and the Communications Manager at Transparify. 

Aaron Erlich is an Assistant Professor of Political Science and Founding Member of the Center for Social and Computations Data Science at McGill University, and previously a Research Consultant at the Caucasus Research Resource Centers in the Tbilisi office.