Monday, December 11, 2017

Evaluation of the Impact of the Agricultural Support Program

CRRC-Georgia carried out a quasi-experimental, post-hoc, mixed methods impact evaluation of the Agricultural Support Program (ASP) between December 2016 and April 2017 in collaboration with the Independent Office of Evaluation (IOE) of the International Fund for Agricultural Development (IFAD). The Agricultural Support Program took place in Georgia between 2010 and 2015. It consisted of two components: 1) Small Scale Infrastructure Rehabilitation and 2) Support for Rural Leasing. For the infrastructure component, the project aimed “to remove infrastructure bottlenecks which inhibit increasing participation of economically active rural poor in enhanced commercialization of the rural economy” according to project documentation. Within the infrastructure component, three types of infrastructure were rehabilitated or built: 1) Rehabilitation of primary and secondary irrigation canals; 2) Rehabilitation of bridges used to bring cattle to pasture; 3) The construction of drinking water infrastructure.


In line with the Internal Office of Evaluation of IFAD’s methodology, impact was assessed across five specific domains. These include: (i) household income and assets; (ii) human and social capital and empowerment; (iii) food security and agricultural productivity; (iv) natural resources, the environment and climate change; and (v) institutions and policies. While the focus of the evaluation is on the rural poverty impact criterion, the performance of the programme has also been evaluated for impact on gender equality and women’s empowerment.

The evaluation assessed not only “if”, but also “how” and “why” the programme has or has not had an impact on selected households and communities in the programme area. To this end, the evaluation team adopted a mixed methods approach including a household survey, focus group discussions, in depth interviews, and key informant interviews. The survey consisted of 3190 interviews, with 1778 interviews in control households and 1412 in treatment households.

In order to test for impact, the team used a quasi-experimental survey design. The driving idea behind quasi-experimental analysis is to use counterfactuals to understand what would have happened in the communities which received interventions had the intervention not taken place. Given that ASP did not make use of randomization, a two staged matching procedure was used to achieve balance on observable variables. First, treated communities were matched with non-treated communities on a number of variables. Second, after data collection households were matched using multivariate matching with genetic weights. Finally, when feasible, a differences in differences approach was used, with changes measured rather than only the 2016 outcome. Regression analyses were then used to estimate effects.

The key findings of the impact analysis include:
  • Indirect beneficiaries of the leasing component – individuals who sold grapes to companies that received leases – had substantively large increases in agricultural incomes;
  • Analyses often suggest little if any impact when it comes to rural poverty. However, context is important. During the project period, ASP was a small part of the very large aid inflows to Georgia, much of which was directed to the area where ASP activities took place. Hence, a lack of significant changes suggests that ASP performed on par with, but not better than other aid projects which took place in control communities;
  • Project outreach in the small scale infrastructure communities was inadequate, resulting in less effective project design and missing an opportunity for the development of human and social capital;
  • The project’s main success within the food security and agricultural productivity impact domain is the increase in amount of land irrigated; the project does not appear to have had any detectable impact on food security;
  • ASP does not appear to have contributed to the sustainable development of the agricultural leasing sector in Georgia;

To read more about the impact evaluation, see the full report, which is available here.

Monday, December 04, 2017

Are Georgians as tolerant as they claim to be?

[Note: This article was co-published with OC-Media and written by Dustin Gilbreath, a Policy Analyst at CRRC-Georgia. The views presented in this article are the author’s alone and do not represent the views of CRRC-Georgia or any related entity.]


On 15 November, the Ministry of Culture announced it would give ‘Georgian tolerance’ the status of intangible cultural heritage. Historically, Georgia may have exhibited relatively high levels of tolerance, with many pointing to the reign of King David the Builder in the 12th century. David is celebrated for presiding over the start of the country’s golden age, and many point to his encouragement of other ethnicities settling in Georgia as a good example of Georgian tolerance.

Yet, recent events in Georgia like the far-right March of Georgians, numerous incidents targeting Georgia’s Muslim community, and the 2013 riots during International Day Against Homophobia suggest Georgia has a ways to go when it comes to tolerance.

Survey data also consistently suggest modern day Georgia lacks tolerance towards minorities of all stripes, though is more tolerant than neighbouring Armenia and Azerbaijan. According to Caucasus Barometer data, Georgians generally disapprove of Georgian women marrying other ethnicities, a common proxy for tolerance in international surveys.

The graph below shows the net approval rating of women marrying other ethnicities between 2009 and 2015. Net approval ratings show whether attitudes are more positive or negative towards an individual or subject overall. The only positive net approval rating is for Russians and only in 2015. Every other net approval rating is less than zero suggesting more Georgians disapprove of women of their ethnicity marrying other ethnicities than approve.  This includes many ethnicities which the Ministry of Culture is presumably celebrating the country’s tolerance towards such as Armenians, Azeris, Abkhaz, Jewish people, and Ossetians.


The data also shows a clear religious bias. The chart below shows the average net approval of women in Georgia marrying ethnicities associated with Christianity and other religions. The graph shows that ethnicities traditionally associated with religions besides Christianity are less approved of by about 20 percentage points.


Note: Ethnicities associated with Christianity in the above graph include Russians, Armenians, Armenians living in Georgia, Ossetians, and Abkhaz. Ethnicities not associated with Christianity include Azeris, Azeris living in Georgia, Jewish people, Kurds, and Turks.

While Georgia lacks in tolerance, when compared with its neighbors in Armenia and Azerbaijan, Georgia is welcoming. The graph below shows the net approvals for each ethnicity on the Caucasus Barometer between 2009 and 2013. The general pattern holds whether or not Armenians and Azerbaijanis attitudes towards each other are taken into account.



Note: Average net approvals were calculated using all ethnicities that were asked about in all three countries as well as the two South Caucasian neighbors of each country. Titular ethnicities were excluded from the calculation. Caucasus Barometer was not carried out in Azerbaijan in 2015.

While Georgia is more tolerant than its neighbors, it still has a long way to go, especially if it wants to match the famed tolerance of King David the Builder.

The data used in this article is available from CRRC-Georgia’s Online Data Analysis Tool.

Monday, November 27, 2017

Perceptions of professionalism, corruption, and nepotism in local government

Professionalism, honesty, and fair competition are important in any institution. Yet, incidents involving corruption, nepotism and/or a lack of professionalism are sometimes reported in the Georgian media when the work of local government bodies is covered. How does the public perceive local government? This blog post describes data from the June 2017 CRRC/NDI survey, which show that a majority of people in Georgia thought that there were problems with nepotism and a lack of professionalism in local government. Moreover, roughly half of the population thought that their local government also faces a problem with corruption.

These assessments vary across different settlement types. The population of Tbilisi and other urban settlements was most likely to think that their local government faces all three problems. A majority of people from rural and ethnic minority settlements agreed that there was a lack of professionalism in their local government, but fewer agreed that there were problems with nepotism and corruption. Notably, as with many other survey questions, a high share of the population of ethnic minority settlements answered “Don’t know”.



People’s perceptions also vary by their level of education. People with higher levels of education agreed more often that their local government has issues with nepotism, professionalism, and corruption.



Note: Answer options to the question “What is the highest level of education you have achieved to date?” were recoded. Answer options “Did not obtain a nine-year certificate”, “Nine-year certificate”, and “Secondary school certificate” were combined into “Secondary or lower”. Answer options “Bachelor’s degree/5-year diploma” and “Any degree above Bachelor’s” were combined into “Higher than secondary”. 

In June 2017, a majority of the population of Georgia agreed that their local government had issues with professionalism and nepotism. Corruption was less often reported to be a problem, though 1/3 of the public (and almost half of the population of Tbilisi) agreed with the statement that there was corruption in their local government.

To explore the data in this blog post more extensively, visit CRRC’s Online Data Analysis tool.


Monday, November 20, 2017

Was the population informed about the constitutional reform in Georgia?

[Note: This blog first appeared in OC-Media. The article was written by Tsisana Khundadze, a Senior Researcher at CRRC-Georgia. The views presented in this article are the author’s alone and do not necessarily reflect the views of CRRC-Georgia, the National Democratic Institute or any other related entity.]

After 10 months of discussions, the parliament of Georgia adopted amendments to the constitution of the country on September 29th and overrode the president’s veto on October 13th, 2017. The most widely discussed amendments are about rules for electing the president, self-governance principles, the definition of marriage, the sale of agricultural land to foreigners, the minimum age of judges and the country’s foreign policy  orientation. Because of the importance of the amendments, one would expect a high level of awareness among the population. However, despite the public meetings held and media coverage of the issue, according to the CRRC/NDI survey from June 2017, a majority of the population of Georgia was not aware of the constitutional reform process.

Survey fieldwork was conducted before the process of amending the constitution finished. Thus, people were asked whether they were aware or not that the State Constitutional Commission had adopted a Draft Revision of the Constitution. Thirty two percent of the population answered they were aware, while 60% stated that they were not. Of those who said they were aware, only 39% said they felt they had enough information about the proposed changes. Moreover, only 6% of people who were aware of the changes said they thought the changes fully reflected citizens’ opinions, and 47% said they partially reflected citizens’ opinions. A third (32%) said the proposed changes did not reflect people’s opinions at all.

People living in the capital were more informed compared to people living outside the capital. Notably, only 13% of people living in ethnic minority settlements reported being aware of the constitutional reform.


Younger people and those with lower levels of education were less aware of the the constitutional reform. Only 26% of people between the ages of 18 and 35 said they were aware. By comparison, 33% of people between the ages of 36 and 55 and 37% of those 56 and older said the same. Similarly, only 19% of people with secondary or lower education were aware of the changes, while 34% of people with secondary technical and 50% of people with tertiary education said they were aware.

Besides differences by age and the level of education, people naming different political parties as closest to them reported being aware of the process at different frequencies. Those who named Georgian Dream - Democratic Georgia, the Alliance of Patriots of Georgia and Bakradze-Ugulava - European Georgia were more informed than people who said that the United National Movement (UNM) or Georgian Labor Party were closest to them. The UNM’s and Labour Party’s supporters were least aware of the process surrounding constitutional amendments.



Awareness of the constitutional reform was low even after public meetings were held to discuss the changes. Younger people, people with secondary or lower education and people living outside Tbilisi were less informed about the process compared to older people, people with tertiary education and people living in the capital. The low level of awareness is especially striking in ethnic minority settlements and among people who named the United National Movement as the party closest to them. As for attitudes towards the changes, a majority of those who were aware felt they did not have enough information about the process and thought that the draft constitution either partially reflected, or did not at all reflect citizens’ opinions.

The data used in this blog post and other survey data is available at our Online Data Analysis portal.


Monday, November 13, 2017

Who should own land in Georgia? How attitudes changed between 2015 and 2017

[Note: This article originally appeared on OC-Media. It was written by Kristina Vacharadze, Programmes Director at CRRC-Georgia. The views presented in the article are the author’s alone, and do not necessarily reflect the views of CRRC-Georgia or the Europe Foundation.]

Georgian parliament recently adopted constitutional amendments. Among the many changes were those regulating the sale of agricultural land. According to the amendments, “Agricultural land, as a resource of special importance, can only be owned by the state, a self-governing entity, a citizen of Georgia, or a union of Georgian citizens.” While the constitution allows for exceptions, which should be regulated by a law yet to be written, it is expected that foreigners will not be allowed to buy agricultural land in Georgia as freely as Georgian citizens. This blog post looks at public opinion about foreigners owning land in Georgia.

A majority of the population (64%) think that land should only be owned by Georgian citizens no matter how they use it, according to the EF/CRRC survey on Knowledge of and Attitudes toward the EU in Georgia (EU survey) conducted in May 2017. This share increased by 21 percentage points since 2015.


Note: The original 11-point scale was recoded into a 5-point scale for the charts in this blog post. Codes 0 and 1 were combined into ‘Only citizens of Georgia should own land in Georgia, no matter how they use this land’; 2 and 3 into ‘2’; 4, 5 and 6 into ‘3’; 7 and 8 into ‘4’ and codes 9 and 10 - into ‘Land in Georgia should be owned by those who will use it in the most profitable way, no matter their citizenship’.  

The rural population is least favourable to the idea of foreign ownership of Georgian land. A large majority (74%) strongly believe that only citizens of Georgia should own land.


The younger population (18-35 years old) is more open towards foreigners owing land in Georgia. Approximately one in five believes that land should be owned by those who will use it in the most profitable way, irrespective of their citizenship. Older people are less open to foreign ownership. Still, in 2017 the proportion of young people who are more open towards foreigners owning land in Georgia dropped by seven percentage points compared to 2015, while the proportion of young people who think that the land should be owned only by Georgian citizens increased by 22 percentage points.


The majority of the population of Georgia do not favour foreigners buying land in the country. Younger people and those living in urban settlements appear more open to the idea of foreign ownership of Georgian land. But the number of those opposing foreign ownership of Georgian land is high and has increased in the past two years. 

Explore the data used in this blog post further using our Online Data Analysis tool.

Friday, November 03, 2017

Taking partly free voters seriously: autocratic response to voter preferences in Armenia and Georgia

Do voters in less than democratic contexts matter or are elections simply facades used to create a veneer of democratic accountability for domestic and international actors? Within the Autocratic Response to Voter Preferences in Armenia and Georgia project, funded by Academic Swiss Caucasus Net, CRRC-Georgia and CRRC-Armenia aimed to help answer this question, at least for Georgia and Armenia. On October 27, Caucasus Survey published the results of the project in a special issue, available here.

In the introduction of the issue, Koba Turmanidze and Matteo Fummagali ask do voters matter in competitive authoritarian regimes and, if so, how? Do their preferences make any difference in the way in which the regime conceives of policies and goes about policy-making? In their article, they argue that they do, and that incumbents take voters seriously. Crucially, the way the regimes respond to policy demand determines their durability in office. The article explains why, despite strong similarities, the political regime ruling Armenia remained stable over the years (from the mid-1990s), whereas the one in Georgia has been unseated on two occasions (2003–2004 and 2012–2013). Evidence confirms that policy-making and voters’ perceptions thereof also play an important role in determining whether a regime collapses or survives. The incumbents collect information on voter preferences, and devise policies in response to them. Policy-making thus matters and is extremely consequential. Paradoxically, however, policy-making makes a difference in counter-intuitive ways. The article concludes that a regime which refrains from making grand promises, or blatantly contradictory or unrealistic ones, has greater chances of surviving than those that set out to transform society, like Saakashvili’s Georgia. Ultimately, such policies backfire on those who launched them.

In the second article in the issue, Dustin Gilbreath and Koba Turmanidze highlight how state capacity volatility and growth affects political survival. Political science has dedicated extensive attention to the determinants of regime change as well as its relation to state capacity. Less work has focused on incumbent political survival and state capacity. Building on selectorate theory (Bueno De Mesquita et al., 2005), the article suggests that the chance of the party of the incumbent remaining in office is partially a function of the capacity of the state they hold power over. However, the authors also hypothesize that state capacity volatility decreases an incumbent’s chances of winning elections. To empirically test these hypotheses, the article uses a cross country statistical analysis complemented by illustrative case studies of policy making from Armenia and Georgia. The analyses support the above two hypotheses, showing that if the incumbent increases state capacity, it increases their chances of staying in office. However, capacity volatility decreases their chances of survival. While Georgian state capacity developed in fits, jumps, and starts, in Armenia state capacity developed at a slow and steady pace for most of its independence. As the aphorism goes, slow and steady wins the race with politicians being thrown out of office in Georgia and the incumbent in Armenia maintaining its power. Based on the analyses presented in the analyses, the authors suggest that a self-defeating game is at work for reformers.

In the third article in the issue, Dustin Gilbreath and Sona Balasanyan take a historic look at election fraud in Armenia and Georgia. In the article they note that elections on unfair playing fields are common, yet election day fraud can result in authoritarians losing office. The freer the environment, the more an authoritarian must rely on means other than election day fraud to retain office, because they are less capable of coercing the population without facing repercussions. Among those other means is cooptation through public policy. A common theme in the special issue is that public policy has been of greater import in Georgia than Armenia. The article makes a contribution to explaining the phenomenon using comparative case studies of election day fraud in Armenia and Georgia over time. To do so, the article uses methods from the field of election forensics to provide a quantitative comparison of the scale of election day fraud in each country’s elections since 2007 using precinct level election results for parliamentary and presidential elections. The test results suggest, as has been widely believed, that Georgia’s elections have had less election day fraud than Armenia’s during this period. This finding provides a theoretical basis to explain why public policy has been a greater concern in Georgia than Armenia.

In the fourth article in the issue, Giorgi Babunashvili argues that while voters are often assumed to be of tertiary importance in less than democratic contexts – the regime can manipulate, buy, or outright steal their votes goes the predominant logic – in reality, voters not only matter but engage in retrospective voting in Georgia, a country with imperfect political competition. Analysis of two waves of nationally representative survey data from 2012 to 2015 supports the retrospective voting theory, with a positive relationship between voter support for the incumbent party and positive assessments of government policies related to socio-economic, democratization, and security issues. Citizens who assess government policies negatively are more prone to voting for opposition candidates or not voting at all compared to those who are more satisfied with the government's performance in Georgia. Notably, these findings are very similar for two governments led by two very different parties in Georgia: the United National Movement (2008–2012) and Georgian Dream – Democratic Georgia (since 2012). Hence, the author concludes that disregarding voters’ preferences has negative consequences for the legitimation and survival of the incumbent.

In the fifth article in this issue, Koba Turmanidze asks: Lie a little or promise a lot? This is a question many politicians face when campaigning before elections. The paper examines whether voters support ambiguous pre-election promises in Armenia and Georgia using an experimental design and if so, what it tells us about accountability mechanisms and a potential accountability trap. The accountability trap emerges when voters cannot hold their elected officials accountable for their promises due to their ambiguity and become disillusioned with political participation. The paper looks at how voters’ expected political behaviour changes in response to randomly assigned types of electoral promises from a hypothetical party. The paper shows a positive effect of ambiguity: if a party makes an ambiguous promise, it will do significantly better in Georgia and at least not worse in Armenia than a party promising a specific policy option. The effect of ambiguity partially explains why parties have been poor at putting forward coherent electoral programs in Armenia and Georgia. More broadly, the findings contribute to understanding the problem of accountability in hybrid regimes, which may lead to representation crises.

In the sixth article in the issue, Rati Shubladze and Tsisana Khundadze point out that voters care about policy, and this is true for democracies as well as hybrid regimes. To show how incumbents’ policy choice influences political continuity and change they look at public policies in Armenia and Georgia from 2004 to 2013. The paper is grounded in Gerschewski’s theoretical framework that views legitimation, repression, and co-optation as the three strategies or pillars of stability in less than democratic regimes. The authors describe each pillar as a set of specific policies designed by ruling parties to gain legitimacy in the eyes of voters, as well as policies aimed at co-optation and/or repression of political opponents. Hence, they demonstrate that the key to the incumbent’s electoral survival is the stabilization process between pillars, i.e. complementary application of policies based on available resources. However, the application of different stabilization strategies is not enough and timing, organization, and balance between pillars are also crucial for maintaining voters’ support for the incumbent. Based on secondary statistical evidence and primary qualitative data analysis, they show how the Armenian government managed to balance the pillars of stability by the effective and well-timed application of different policies, while the government of Georgia failed to use relevant pillars of stabilization when one of the pillars did not work to the incumbent’s advantage.

Overall, the issue makes the case that voters - even in less than democratic contexts - matter.  To view the articles, click on the links above or here for the entire issue.



Monday, October 30, 2017

Georgian public increasingly unaware of what the European Union Monitoring Mission does

[Note: This article originally appeared at OC-Media. The article was written by Dustin Gilbreath, a Policy Analyst at CRRC-Georgia. The views presented in this article are the author’s alone and do not necessarily reflect the views of CRRC-Georgia or Europe Foundation.]

As much as 81% of the population of Georgia doesn’t know what the European Union Monitoring Mission (EUMM) does, according to the 2017 Knowledge of and Attitudes towards the European Union in Georgia survey funded by Europe Foundation and implemented by CRRC-Georgia. This lack of knowledge has increased over time, as has the prevalence of incorrect information about the EUMM’s mission. This represents a missed opportunity for the EU’s communications in Georgia.

On the survey, respondents were asked, “What does the European Union Monitoring Mission do in Georgia?” A large plurality of the population (41%) reported they did not know what the EUMM does. The second most common response (25%) was “supports the implementation of democratic and market oriented reforms,” an incorrect answer. The third most common response (19%) was “supports the stabilization of the situation in the areas affected by the August 2008 war,” the EUMM’s actual mission.

The share of the public aware of what the EUMM does has declined over time. While in 2009, 39% of the population knew what the EUMM did, 19% did in 2017. This decline may stem from the relatively high salience of the Monitoring Mission in the years immediately following the 2008 August War with Russia, although no data exists which would confirm this.

Besides the decline in knowledge of what the EUMM does, inaccurate information about the organization has become more prevalent. While only 24% of the public gave an inaccurate answer to the question in 2009, 38% did in 2017. Notably, there was a large increase in don’t know responses in 2015 and a sizable decline in 2017. Rather than an increase in correct responses in 2017, however, the data suggests that a lack of knowledge was replaced by incorrect information.



The lack of knowledge about the EUMM is most pronounced in ethnic minority settlements, with only 2% of individuals in minority communities correctly responding to the question. The lack of knowledge in minority settlements should come as no surprise given that surveys in these communities regularly have high rates of don’t know responses.

In contrast, those living in rural settlements with a predominantly ethnic Georgian population provide the correct response most often. The fact that the rural population is more informed than urban populations may stem from the EUMM’s rural presence. While the organization has offices in four urban settlements – Tbilisi, Mtskheta, Zugdidi, and Gori – they regularly patrol the rural areas surrounding the administrative boundary lines with Abkhazia and South Ossetia.

Although the public is increasingly unaware of what the EUMM does, about 2/3 of people who provided incorrect answers to the question about the EUMM’s mission would like to have more information about the EU. About a quarter even want information specifically about the EU’s role in resolving Georgia’s territorial conflicts.

The lack of knowledge of what the European Union Monitoring Mission does in Georgia may represent a missed opportunity for the EU. While no data is available about the attitudes of people who have had contact with the EUMM, previous research in Georgia has suggested that individuals contacted by NGOs report greater trust in them. The EUMM, given its public service mission, may receive a comparable boost from contact with the public. Hence, the EU should consider increasing its outreach and communications related to the EUMM.