Monday, June 26, 2017

CRRC’s Fifth Annual Methodological Conference: In Search of Methodological Innovation

CRRC’s fifth annual Methodological Conference took place on June 23 and 24, 2017 in Tbilisi. This year the conference’s focus was on policy analysis in the South Caucasus, and the search for methodological innovation. Over 50 participants representing institutions in the United States, United Kingdom, Georgia, Azerbaijan, Armenia, Russia, and Canada attended.

Alexis Diamond of the Keck Graduate Institute (KGI), San Francisco gave an opening address for the conference titled: Deliberate ignorance: The dangers of knowing too much too soon. The talk covered a wide range of issues in evaluation, however, emphasis was placed on the importance of honest evaluation.
The first day of the conference had four sessions, with papers on a wide variety of subjects from the geographies of polarization and inequality in Tbilisi to a field experiment on marshrutka safety and a machine learning approach to profiling tax awareness in Armenia. 
The opening slide of David Sichinava’s presentation on Spatial Patterns of Emerging Inequalities in Tbilisi, Georgia.
The second day of the conference was dedicated to workshops. Alexey Levinson of the Levada Center, Moscow lead a workshop on open-ended group discussions and Aaron Erlich of McGill University discussed the fundamentals of multiple imputation. The conference also included workshops on case studies in public health, web surveys, and synthetic controls.
Aaron Erlich discussing why and when to use multiple imputation.
For more information, the full conference program can be accessed here.

Monday, June 19, 2017

Back to the USSR? How poverty makes people nostalgic for the Soviet Union

A recent CRRC/NDI survey asked whether the dissolution of the Soviet Union was a good or bad thing for Georgia. People’s responses were split almost evenly: 48% reported that the dissolution was a good thing, whereas 42% said it was a bad thing for the country. Such a close split raised questions in the media about why people took one view or another.

While it is tempting to explain assessments of a past event, such as the dissolution of the Soviet Union, using people’s attitudes towards foreign policy issues, this blog post only looks at respondents’ socio-demographic and economic characteristics and some reported behaviors that could potentially shape their attitudes. Specifically, we look at the impact of gender, age, education, ability to speak English and Russian, frequency of internet use, settlement type and the number of durable goods a household possesses, out of the ten durables the survey asked about: a refrigerator, color TV, smartphone, tablet computer, car, air conditioner, automatic washing machine, personal computer, hot water, and central heating. We interpret the number of durables owned as a measure of the households’ economic status. Surely, this measure is not perfect and gives us only partial information about the household’s economic conditions. However, this is the best available measure from this particular survey, provided that many people do not like reporting their income or expenditures, or do not provide accurate information on these.

The chart below shows the results of a logistic regression model which predicts the odds of a respondent saying that the dissolution of the Soviet Union was a good thing for Georgia. The dots on the chart indicate point estimates for each independent variable, and the lines show 95% confidence intervals. If a line does not cross the vertical red line, we are 95% confident that the variable has an impact on the dependent variable, i.e. the belief that the dissolution of the Soviet Union was a good thing for Georgia. The further a horizontal line from the vertical red line, the larger the effect of the variable.

The model shows that gender and the ability to speak either English or Russian do not influence people’s assessments of the dissolution of the Soviet Union. As one might expect, age has a significant, negative impact: the older a person is, the lower is the probability that s/he will express a positive attitude towards the dissolution of the USSR. Education, frequency of internet use and possession of durables have the opposite impact: people with tertiary education are more likely to assess the dissolution positively than people with less than tertiary education. Likewise, people, who use the internet at least once a week assess the dissolution more positively than people who use the internet less often or never. Also, the more durables a household owns, the higher the probability of assessing the dissolution of the Soviet Union as a positive event for Georgia.

As expected, settlement type also matters: the chart shows the effect of living in Tbilisi, other large towns, predominantly Georgian-speaking rural settlements and ethnic minority settlements, which are compared to small towns – the reference category. Large towns include six cities with more than 40 thousand people, whereas smaller urban settlements are grouped into a small town category. We define an ethnic minority settlement as a location in which 40% or more of the inhabitants are ethnic minorities. Normally, these are towns and villages with large Armenian or Azerbaijani populations in the Kvemo Kartli, Samtskhe-Javakheti and Kakheti regions.

While residents of large towns and rural settlements have similar opinions about the dissolution of the Soviet Union as residents of small towns, Tbilisi residents are more likely to assess the dissolution positively. In contrast, those living in minority settlements tend to assess the dissolution negatively.
Based on the above, we conclude that age, education, frequency of internet use, possession of durables and settlement type influence an individual’s assessment of the dissolution of the Soviet Union. As a next step, we tested whether age, education, frequency of internet use, and possession of durables influence individuals’ attitudes differently in different settlement types.

The analysis shows that the impact of age, education and internet usage does not vary by settlement type. However, we observe a very different picture in the case of household possessions: possessing more durables increases the probability of positive assessment of the dissolution of the USSR in all settlement types except for (non-minority) villages and small towns. Its impact is largest, however, for residents of ethnic minority settlements. If an individual living in such a settlement has no durables, his or her probability of assessing the dissolution of the Soviet Union positively is below 20%. However, as the number of durables in the household increases, the probability of a positive assessment increases nearly linearly, and exceeds 60% when the household owns all ten items asked about on the survey.

Hence, we conclude that age, education, settlement type, and economic conditions significantly influence people’s assessments of the dissolution of the Soviet Union. The impact of a household’s economic situation is largest in ethnic minority settlements. Therefore, economic deprivation, arguably caused by and interrelated with a number of other factors, seems to be the most important driver of negative assessments of the dissolution, rather than minority status per se.  

To have a closer look at the CRRC/NDI data, visit CRRC’s Online Data Analysis tool.

Monday, June 12, 2017

Most households in Georgia report limiting food consumption, despite economic growth

According to the World Bank, GDP in Georgia increased from USD 10.1 billion to USD 13.9 billion between 2009 and 2015. Despite this growth, according to CRRC’s Caucasus Barometer survey (CB), the share of those who reported not having enough money to buy food on at least a weekly basis did not decrease between 2011 and 2015. This blog post shows how this finding differs by settlement type and reported household income.

As the chart below shows, the general picture did not change between 2011 and 2015. Only about one third of the population claims it never happened during the 12 month prior to the survey that they did not have enough money to buy food they or their family needed. More than one third report encountering such difficulties periodically and about a quarter monthly or more often.


Note: Answer options “Every day” and “Every week” have been combined for the charts in this blog post.

Taking into account the margin of error, the share of people who reported not having enough money to buy food every week or more often is approximately the same in different settlement types. Importantly, the most common response in Tbilisi and the second most common response in other urban settlements is “Never”. This answer is, however, reported by less than half of the population of these settlement types.

Note: Answer options “Don’t know” and “Refuse to answer” were excluded from the analysis.
                       
A lack of money for food logically suggests a low income. The chart below shows that the higher the reported household income, the higher the share of the population reporting never being in a situation when they did not have enough money for food. According to CB 2015, 61% of the population reported their household income was less than USD 250 the month prior to the survey.


CB data show that a large share of households in Georgia have financial difficulties supporting their families’ primary needs and a majority struggle with not having enough money for food at least some of the time.

To have a closer look at the Caucasus Barometer data, visit CRRC’s Online Data Analysis tool.

Tuesday, June 06, 2017

Georgia: Disapproval Rising for NATO Membership

[Note: This piece was originally published on Eurasianet. Dustin Gilbreath is a policy analyst at CRRC-Georgia. Rati Shubladze is a researcher at CRRC-Georgia. The views expressed in this article do not necessarily reflect the views of either CRRC-Georgia or the National Democratic Institute.]

NATO recently recognized Georgia’s contributions to peacekeeping missions from Afghanistan to Kosovo by holding a session of the alliance’s parliamentary assembly in Tbilisi in late May. The occasion reinforced the hopes of Georgian leaders that their country can one day soon gain admission to NATO. However, polling in the lead up to NATO’s parliamentary assembly also sheds light on a trend that could potentially hinder its membership bid.

Public support in Georgia for the country’s NATO membership bid remains strong. A recent survey CRRC-Georgia carried out for the National Democratic Institute shows that 68 percent of Georgians support the government’s goal of joining the alliance. If Georgia were a NATO member, this would be the third highest level of support of any member state polled in a recent Pew Research Center survey.

Yet, CRRC and NDI’s data also shows that disapproval with the prospect of membership is rising. Back in 2012, roughly a quarter of the public was uncertain over whether the country should join NATO; since 2015, however, only about one in 10 have reported uncertainty. Over the same period, disapproval of NATO membership doubled from about one in 10 Georgians to roughly one in five.

This trend has at least two potential explanations.

First, people who used to report that they are not sure about membership might have always been opposed to the Alliance. Rather than telling interviewers this, they felt social pressure not to say so, because they perceived NATO support to be popular in Georgia. This phenomenon, being shy about reporting unpopular opinions to survey interviewers, is common, and is known as social desirability bias.

If this explanation is correct, then the shift from uncertainty in response to disapproval is a sign of a trend in Georgian society and its foreign policy discourse: anti-Euro Atlantic views are more widely accepted or at least perceived to be more socially acceptable than in years past. Over the past couple of years, politicians have expressed less confrontational views towards Russia, at least when compared with the virulently anti-Russian rhetoric of former president Mikheil Saakashvili and his United National Movement, which lost its parliamentary majority in 2012. This change in discourse might contribute to the trend, making it more widely acceptable to express views that are not pro-Western. While beyond the scope of this article, Russian propaganda too could be playing some role.

A second possible explanation is that a significant number of those who were previously undecided are now making up their minds: no longer sitting on the fence, they have decided that the actual or potential costs of NATO membership are too great, or the chance of NATO membership too low, to make the required sacrifices.

Georgia is a small country and, even in absolute terms, it contributed more soldiers to NATO’s mission in Afghanistan than any other non-member state. This contribution has not come without a cost: over 30 Georgian soldiers have died in Afghanistan and hundreds have been wounded.

The potential for NATO membership to incite Russia’s ire weighs heavily in the minds of those who disapprove. When those who reported opposing Georgia’s NATO bid were asked why they disapprove, the most common response was that it will cause conflict with Russia.

Despite Georgia’s sacrifices, membership in the Alliance seems distant to a majority of Georgians. Since the 2008 Bucharest Summit Declaration, which stated that Georgia and Ukraine would someday become members of the Alliance, a Membership Action Plan – a first step towards membership – has proven elusive. This is reflected in public opinion about when Georgia will join NATO: 16 percent think the country will never join, and an additional 38 percent are uncertain if or when the country will be offered membership. A majority of those who are uncertain about a membership date favor Georgia’s NATO bid.

Even though disapproval of Georgia’s NATO bid may be rising, the head of the NATO Parliamentary Assembly has suggested that Georgia is more prepared for membership than even some member states. Notably, Georgian military expenditure has consistently exceeded 2 percent of GDP, the level required of NATO members, despite the fact that only five member states meet this spending target. On top of this, a full 80 percent of those polled think that military spending should either stay the same or increase.

If the Alliance is dedicated to its 2008 Bucharest commitments, it should make its intentions clear to the Georgian public. The lack of a clear signal from the Alliance seems likely to only keep stoking uncertainty and disapproval of members among the Georgian public.


Monday, May 29, 2017

Awareness of EU aid and support for EU membership in Georgia

The EU provides a wide variety of aid to Georgia. Within the European Neighbourhood and Partnership Instrument (ENPI) alone, EUR 452.1 million was allocated for Georgia in 2007-2013. What, if any, role does this aid play in influencing the population’s attitudes towards the European Union? This blog post looks at the awareness of the population of Georgia about the EU’s aid to the country, on the one hand, and support for Georgia’s membership in the EU, on the other hand, using the 2015 CRRC/EF survey on Knowledge of and Attitudes towards the EU in Georgia (EU survey). The findings suggest that those who are more aware of EU aid are also more likely to support Georgia’s membership in the EU.

About 1/10th (13%) of the population of Georgia think the EU does not assist Georgia in any way, while about a third (31%) could not or would not answer the question, “In your opinion, what are the main types of aid the EU currently provides to Georgia?” Of the 56% who named a specific type of aid, humanitarian aid and investment in Georgia’s economy were the most frequent answers.

Note: A show card was used. The answers do not add up to 100% since respondents could choose up to three answer options. 

This question was recoded for further analysis. In a new variable, “Does the EU provide any aid to Georgia?”, all answers where a certain type of aid was mentioned were combined into the category “The EU provides aid,” while options “[The EU] does not provide any aid”, “Don’t know” and “Refuse to answer” were combined into the category “Don’t know/Does not provide aid.”

As the 2015 EU Survey report and CRRC’s previous blog posts have highlighted, in 2015, 61% of the population of Georgia reported that they would vote for the country’s membership in the EU, if a referendum was held tomorrow. Seventy-five percent of those saying that the European Union provides any type of aid to Georgia report that they would vote for EU membership. Only 44% of those saying the EU does not provide aid or answered “Don’t know” report the same.


The 56% of the population who named a specific type of aid that, in their opinion, Georgia receives from the EU, were then asked which groups benefit most from this aid. Politicians and high level officials in Georgia were named most often as the group that benefits most from EU aid to the country.


Note: The question was asked only to those who reported that the EU provides aid to Georgia. A show card was used for this question. The answers do not add up to 100% since respondents could choose up to two answer options. 

These findings suggest that awareness about EU aid to Georgia matters when it comes to people’s support for Georgia’s membership in the EU. Those who believe that the EU provides specific types of aid to Georgia tend to report more often that they would vote for Georgia’s membership in the EU.

To learn more about the population’s support for Georgia’s membership in the EU, visit CRRC’s Online Data Analysis tool, or take a look at some of CRRC-Georgia’s blog posts on the subject, here.

Thursday, May 25, 2017

Who is more intelligent and sincere? CRRC interviewers’ assessments

It has been only about two decades since pollsters started using paradata, including interviewer assessments of the conducted interviews and respondents’ behavior and attitudes. Such assessments are collected in the process of most of CRRC-Georgia’s surveys. Immediately after each interview, interviewers are asked to assess various aspects of the interview, including respondents’ sincerity and intelligence. Using CRRC’s 2015 Caucasus Barometer (CB) data for Armenia and Georgia, this blog post explores how such assessments differ depending on respondents’ demographic characteristics and their households’ economic situations.

It is important to highlight that these assessments can hardly provide reliable information about the actual intelligence or sincerity of respondents, and there is no doubt that interviewers’ perceptions of intelligence are very subjective, thus they cannot be attributed to any objective characteristics. Still, when analyzed properly, they can tell us a lot about a wide range of issues, from the process of interviews to the dominant stereotypes and biases in a given society.

In the CB 2015 assessment forms, interviewers were asked to rate respondents’ intelligence. In both Armenia and Georgia, interviewers tend to rate the intelligence of people who live in the capital and other urban settlements higher than the intelligence of those who live in villages.


 Note: The original 5-point scale was recoded for the charts in this blog post, with answer options “Intelligent” and “Very intelligent” combined into the category “Intelligent” and answer options “Not at all intelligent” and “Not very intelligent” combined into the category “Not intelligent”. Responses “Don’t know” and “Refuse to answer” were excluded from the analysis in the charts in this blog post.

Although, overall the respondents are reported to be rather sincere during the interviews, interviewers report the answers of people living in the capital and other urban settlements to be slightly more sincere compared to the answers of people living in villages.

Note: The original 11-point scale was recoded for the charts in this blog post, with codes 0 through 3 combined into the category “Not sincere”, codes 4 through 6 combined into “Average” and codes 7 through 10 combined into the category “Sincere”. 

There are even more pronounced differences by respondents’ level of education. Interviewers rate people with a lower level of formal education both as less intelligent and sincere. In both countries, people who report having secondary technical education or secondary or lower education are twice as likely to be placed in the category “Average” than those with a higher than secondary education.
 Note: Answer options “No primary education”, “Primary education”, “Incomplete secondary education” and “Secondary education” were grouped into the category “Secondary or lower”; “Incomplete higher education”, “Completed higher education” and “Post-graduate degree” were grouped into the category “Higher than secondary”. 

In contrast to the assessments of intelligence, interviewers’ assessments of respondents’ sincerity vary less by the respondents’ level of education. Both these assessments, however, are very similar in Armenia and Georgia.


Interviewers’ assessments of respondents’ intelligence and sincerity also differ according to the reported economic situation of the respondent’s household. Interviewers tend to perceive those who are better-off economically as more intelligent. For example, interviewers report people living in households that can afford to buy expensive durables to be intelligent almost three times more often than people who do not have enough money for food, in both Armenia and Georgia.

Note: Answer options “We can afford to buy some expensive durables [like a refrigerator or washing machine]” and “We can afford to buy anything we need” were grouped into the category “Enough money for expensive durables”.

Those who report being in the best economic situation tend to be rated as sincere more often than people in all other groups.
Further research is needed to find out to what extent interviewer assessments reflect the actual situation, i.e. whether those who are better off, or urban dwellers, are actually more intelligent or sincere, and vice versa. The findings presented in this blog post show that interviewers have rather strong opinions about what groups of respondents are intelligent and sincere, and the assessments are very similar in Armenia and Georgia. Interestingly, there is a high positive correlation between the assessments of respondents’ intelligence and sincerity in both countries, suggesting another interesting topic for further research, namely, to what extent intelligence and sincerity are seen by interviewers as interrelated characteristics.

To find out more about CRRC’s Caucasus Barometer and other surveys, check out our online data analysis tool.

Monday, May 22, 2017

Air Pollution in Tbilisi: What the Data Says

[Note: This is a guest blog post by Dr. Hans Gutbrod, the former Director of CRRC.]

With the recent debate on traffic safety, it may also be a good time to highlight the issue of air pollution in Tbilisi – especially as municipal elections are coming up, and citizens (and candidates) may ask themselves what issues the election should focus on.

The data is clear: citizens care about pollution and the environment. In the June 2016 survey that CRRC conducted for NDI-Georgia, pollution was seen as a key concern for residents of Tbilisi. In the capital, 38% named it as one of the most important infrastructural issues, placing it ahead of all other issues related to infrastructure. In general, all generations care, but the young a bit more. In Tbilisi, about 40% of those under 36 years of age mentioned pollution as an issue compared to 32% of those 56 and older.

Politically, the topic also seems to resonate, though less strongly with voters who tend towards the big parties. Only 33% of Georgian Dream voters and 35% of United National Movement voters reported that pollution was the number one infrastructural issue in Tbilisi (Note: the party landscape has changed since the survey). This may in part be a result of the small sample size once you do crosstabs, but could also suggest that the environment and pollution may be an issue to mobilize and rally voters around. Supporting this contention is the fact that 40% of undecided voters in Tbilisi named the issue as the most important one in Tbilisi.

Yet how bad is the situation really? It is not so easy to find out. The government does collect data at three measuring stations that have been donated by Japan, but it is made available one day late, in PDFs, and even those who have interpreted lots of data will need a significant amount of time to decipher what is going on.

A very quick glance on some random days in February suggests that the pollution in Tbilisi repeatedly exceeds limits that are considered healthy – often by a multiple.


Whatever the policy prescription, one sensible next step for citizens and parties to demand, and for the government to take, is to make this data accessible straightaway, live. Having the data, citizens could decide on what to expose themselves to, when. To have a good debate on policy, we need good data. Thus, a sensible suggestion to politicians whenever and wherever you meet them is to request that public pollution data be made available in real time.