CRRC’s Fifth Annual Methodological Conference: In Search of Methodological Innovation

CRRC’s fifth annual Methodological Conference took place on June 23 and 24, 2017 in Tbilisi. This year the conference’s focus was on policy analysis in the South Caucasus, and the search for methodological innovation. Over 50 participants representing institutions in the United States, United Kingdom, Georgia, Azerbaijan, Armenia, Russia, and Canada attended.

Alexis Diamond of the Keck Graduate Institute (KGI), San Francisco gave an opening address for the conference titled: Deliberate ignorance: The dangers of knowing too much too soon. The talk covered a wide range of issues in evaluation, however, emphasis was placed on the importance of honest evaluation.

The first day of the conference had four sessions, with papers on a wide variety of subjects from the geographies of polarization and inequality in Tbilisi to a field experiment on marshrutka safety and a machine learning approach to profiling tax awareness in Armenia. 

The opening slide of David Sichinava’s presentation on Spatial Patterns of Emerging Inequalities in Tbilisi, Georgia.

The second day of the conference was dedicated to workshops. Alexey Levinson of the Levada Center, Moscow lead a workshop on open-ended group discussions and Aaron Erlich of McGill University discussed the fundamentals of multiple imputation. The conference also included workshops on case studies in public health, web surveys, and synthetic controls.

Aaron Erlich discussing why and when to use multiple imputation.

For more information, the full conference program can be accessed here.

Back to the USSR? How poverty makes people nostalgic for the Soviet Union

A recent CRRC/NDI survey asked whether the dissolution of the Soviet Union was a good or bad thing for Georgia. People’s responses were split almost evenly: 48% reported that the dissolution was a good thing, whereas 42% said it was a bad thing for the country. Such a close split raised questions in the media about why people took one view or another.

While it is tempting to explain assessments of a past event, such as the dissolution of the Soviet Union, using people’s attitudes towards foreign policy issues, this blog post only looks at respondents’ socio-demographic and economic characteristics and some reported behaviors that could potentially shape their attitudes. Specifically, we look at the impact of gender, age, education, ability to speak English and Russian, frequency of internet use, settlement type and the number of durable goods a household possesses, out of the ten durables the survey asked about: a refrigerator, color TV, smartphone, tablet computer, car, air conditioner, automatic washing machine, personal computer, hot water, and central heating. We interpret the number of durables owned as a measure of the households’ economic status. Surely, this measure is not perfect and gives us only partial information about the household’s economic conditions. However, this is the best available measure from this particular survey, provided that many people do not like reporting their income or expenditures, or do not provide accurate information on these.

The chart below shows the results of a logistic regression model which predicts the odds of a respondent saying that the dissolution of the Soviet Union was a good thing for Georgia. The dots on the chart indicate point estimates for each independent variable, and the lines show 95% confidence intervals. If a line does not cross the vertical red line, we are 95% confident that the variable has an impact on the dependent variable, i.e. the belief that the dissolution of the Soviet Union was a good thing for Georgia. The further a horizontal line from the vertical red line, the larger the effect of the variable.

The model shows that gender and the ability to speak either English or Russian do not influence people’s assessments of the dissolution of the Soviet Union. As one might expect, age has a significant, negative impact: the older a person is, the lower is the probability that s/he will express a positive attitude towards the dissolution of the USSR. Education, frequency of internet use and possession of durables have the opposite impact: people with tertiary education are more likely to assess the dissolution positively than people with less than tertiary education. Likewise, people, who use the internet at least once a week assess the dissolution more positively than people who use the internet less often or never. Also, the more durables a household owns, the higher the probability of assessing the dissolution of the Soviet Union as a positive event for Georgia.

As expected, settlement type also matters: the chart shows the effect of living in Tbilisi, other large towns, predominantly Georgian-speaking rural settlements and ethnic minority settlements, which are compared to small towns – the reference category. Large towns include six cities with more than 40 thousand people, whereas smaller urban settlements are grouped into a small town category. We define an ethnic minority settlement as a location in which 40% or more of the inhabitants are ethnic minorities. Normally, these are towns and villages with large Armenian or Azerbaijani populations in the Kvemo Kartli, Samtskhe-Javakheti and Kakheti regions.

While residents of large towns and rural settlements have similar opinions about the dissolution of the Soviet Union as residents of small towns, Tbilisi residents are more likely to assess the dissolution positively. In contrast, those living in minority settlements tend to assess the dissolution negatively.
Based on the above, we conclude that age, education, frequency of internet use, possession of durables and settlement type influence an individual’s assessment of the dissolution of the Soviet Union. As a next step, we tested whether age, education, frequency of internet use, and possession of durables influence individuals’ attitudes differently in different settlement types.

The analysis shows that the impact of age, education and internet usage does not vary by settlement type. However, we observe a very different picture in the case of household possessions: possessing more durables increases the probability of positive assessment of the dissolution of the USSR in all settlement types except for (non-minority) villages and small towns. Its impact is largest, however, for residents of ethnic minority settlements. If an individual living in such a settlement has no durables, his or her probability of assessing the dissolution of the Soviet Union positively is below 20%. However, as the number of durables in the household increases, the probability of a positive assessment increases nearly linearly, and exceeds 60% when the household owns all ten items asked about on the survey.

Hence, we conclude that age, education, settlement type, and economic conditions significantly influence people’s assessments of the dissolution of the Soviet Union. The impact of a household’s economic situation is largest in ethnic minority settlements. Therefore, economic deprivation, arguably caused by and interrelated with a number of other factors, seems to be the most important driver of negative assessments of the dissolution, rather than minority status per se.  

To have a closer look at the CRRC/NDI data, visit CRRC’s Online Data Analysis tool.

Most households in Georgia report limiting food consumption, despite economic growth

According to the World Bank, GDP in Georgia increased from USD 10.1 billion to USD 13.9 billion between 2009 and 2015. Despite this growth, according to CRRC’s Caucasus Barometer survey (CB), the share of those who reported not having enough money to buy food on at least a weekly basis did not decrease between 2011 and 2015. This blog post shows how this finding differs by settlement type and reported household income.

As the chart below shows, the general picture did not change between 2011 and 2015. Only about one third of the population claims it never happened during the 12 month prior to the survey that they did not have enough money to buy food they or their family needed. More than one third report encountering such difficulties periodically and about a quarter monthly or more often.

Note: Answer options “Every day” and “Every week” have been combined for the charts in this blog post.

Taking into account the margin of error, the share of people who reported not having enough money to buy food every week or more often is approximately the same in different settlement types. Importantly, the most common response in Tbilisi and the second most common response in other urban settlements is “Never”. This answer is, however, reported by less than half of the population of these settlement types.

Note: Answer options “Don’t know” and “Refuse to answer” were excluded from the analysis.

                       
A lack of money for food logically suggests a low income. The chart below shows that the higher the reported household income, the higher the share of the population reporting never being in a situation when they did not have enough money for food. According to CB 2015, 61% of the population reported their household income was less than USD 250 the month prior to the survey.

CB data show that a large share of households in Georgia have financial difficulties supporting their families’ primary needs and a majority struggle with not having enough money for food at least some of the time.

To have a closer look at the Caucasus Barometer data, visit CRRC’s Online Data Analysis tool.

Georgia: Disapproval Rising for NATO Membership

[Note: This piece was originally published on Eurasianet. Dustin Gilbreath is a policy analyst at CRRC-Georgia. Rati Shubladze is a researcher at CRRC-Georgia. The views expressed in this article do not necessarily reflect the views of either CRRC-Georgia or the National Democratic Institute.]

NATO recently recognized Georgia’s contributions to peacekeeping missions from Afghanistan to Kosovo by holding a session of the alliance’s parliamentary assembly in Tbilisi in late May. The occasion reinforced the hopes of Georgian leaders that their country can one day soon gain admission to NATO. However, polling in the lead up to NATO’s parliamentary assembly also sheds light on a trend that could potentially hinder its membership bid.

Public support in Georgia for the country’s NATO membership bid remains strong. A recent survey CRRC-Georgia carried out for the National Democratic Institute shows that 68 percent of Georgians support the government’s goal of joining the alliance. If Georgia were a NATO member, this would be the third highest level of support of any member state polled in a recent Pew Research Center survey.

Yet, CRRC and NDI’s data also shows that disapproval with the prospect of membership is rising. Back in 2012, roughly a quarter of the public was uncertain over whether the country should join NATO; since 2015, however, only about one in 10 have reported uncertainty. Over the same period, disapproval of NATO membership doubled from about one in 10 Georgians to roughly one in five.

This trend has at least two potential explanations.

First, people who used to report that they are not sure about membership might have always been opposed to the Alliance. Rather than telling interviewers this, they felt social pressure not to say so, because they perceived NATO support to be popular in Georgia. This phenomenon, being shy about reporting unpopular opinions to survey interviewers, is common, and is known as social desirability bias.

If this explanation is correct, then the shift from uncertainty in response to disapproval is a sign of a trend in Georgian society and its foreign policy discourse: anti-Euro Atlantic views are more widely accepted or at least perceived to be more socially acceptable than in years past. Over the past couple of years, politicians have expressed less confrontational views towards Russia, at least when compared with the virulently anti-Russian rhetoric of former president Mikheil Saakashvili and his United National Movement, which lost its parliamentary majority in 2012. This change in discourse might contribute to the trend, making it more widely acceptable to express views that are not pro-Western. While beyond the scope of this article, Russian propaganda too could be playing some role.

A second possible explanation is that a significant number of those who were previously undecided are now making up their minds: no longer sitting on the fence, they have decided that the actual or potential costs of NATO membership are too great, or the chance of NATO membership too low, to make the required sacrifices.

Georgia is a small country and, even in absolute terms, it contributed more soldiers to NATO’s mission in Afghanistan than any other non-member state. This contribution has not come without a cost: over 30 Georgian soldiers have died in Afghanistan and hundreds have been wounded.

The potential for NATO membership to incite Russia’s ire weighs heavily in the minds of those who disapprove. When those who reported opposing Georgia’s NATO bid were asked why they disapprove, the most common response was that it will cause conflict with Russia.

Despite Georgia’s sacrifices, membership in the Alliance seems distant to a majority of Georgians. Since the 2008 Bucharest Summit Declaration, which stated that Georgia and Ukraine would someday become members of the Alliance, a Membership Action Plan – a first step towards membership – has proven elusive. This is reflected in public opinion about when Georgia will join NATO: 16 percent think the country will never join, and an additional 38 percent are uncertain if or when the country will be offered membership. A majority of those who are uncertain about a membership date favor Georgia’s NATO bid.

Even though disapproval of Georgia’s NATO bid may be rising, the head of the NATO Parliamentary Assembly has suggested that Georgia is more prepared for membership than even some member states. Notably, Georgian military expenditure has consistently exceeded 2 percent of GDP, the level required of NATO members, despite the fact that only five member states meet this spending target. On top of this, a full 80 percent of those polled think that military spending should either stay the same or increase.

If the Alliance is dedicated to its 2008 Bucharest commitments, it should make its intentions clear to the Georgian public. The lack of a clear signal from the Alliance seems likely to only keep stoking uncertainty and disapproval of members among the Georgian public.