Monday, May 27, 2019

Does our algorithm still work?

Within the Russian Propaganda Barometer Project, funded by USAID through EWMI’s ACCESS program, CRRC-Georgia created a model, using a k-nearest neighbors algorithm, which attempts to predict whether a person falls into one of three groups: consistently pro-Western; anti-Western; or neither and potentially at-risk of being influenced towards an anti-Western foreign policy position. The model used data from NDI and CRRC’s polling between 2008 and July 2018.  It included variables for age, education level, settlement type, and when the survey was conducted.

In essence, this was done to see whether it is possible to guess a person’s foreign policy preferences using the aforementioned variables. The model successfully predicted people’s status 64% of the time, as described in this policy brief. This was a nine percentage point improvement over always guessing the most common status.

But, as is well known, social scientists can play around with data to make themselves look good. In this regard, one way of understanding how well a model works is whether it can predict observations that are inaccessible to the researcher at the time of the development of the model i.e. does the model make accurate predictions about people it does not yet have information about?

To test the model, CRRC-Georgia used the same algorithm to predict people’s status on the December 2018 and April 2019 NDI survey. Neither dataset was available to the researcher at the time the model was developed.

The results suggest the model works in near identical form, predicting 65% of responses accurately. As in the policy brief, the new data suggest that people in predominantly ethnic minority settlements, people with lower levels of education, and older people are significantly more likely to be at risk of being influenced by anti-Western propaganda in Georgia.


These results lead to a number of conclusions. First, the model does appear to work. In essence, it can guess someone’s foreign policy position correctly two thirds of the time if you know the type of settlement they live in, age, and education level.  Second, it re-affirms the recommendation in the policy brief that people working towards countering anti-Western propaganda in Georgia should prioritize working with ethnic minority communities; people who are not so young; and those with lower levels of education. At some level, these groups are relatively difficult for NGOs to reach. NGOs often focus on working with young people and the highly educated. Moreover, NGOs rarely have the capacity to work in Armenian and Azeri communities, aside from the organizations from those communities.

Despite the challenge in reaching these populations, the need is clear, and if Georgia is to prevent anti-Western propaganda from dividing society actors should work to counter Russian propaganda efforts among these groups.

The views expressed in this blog post do not represent the views of EWMI, USAID, or any related entity.

Replication code for the above data analysis is available here.


Monday, May 20, 2019

Grit among young people in Georgia

Angela Duckworth’s concept grit has gained a great deal of attention in recent years. Grit, described as some combination of perseverance and passion, has gained this attention, because the data suggest it is associated with a number of positive outcomes like employment and completion of education. In 2018, CRRC-Georgia measured the grit of over 2500 young people (15-35) within a baseline evaluation for World Vision’s SAY YES Skills for Jobs project (funded by the European Union within EU4YOUTH program) which is taking place in Mtskheta, Akhaltsikhe, Adigeni, Kutaisi, Zestaponi, Bagdati, Senaki, and Zugdidi. The data suggest that grit is good predictor of positive outcomes in Georgia as is it is in other contexts.

The grit scale is made up of 12 questions, measured on a five point scale, which were asked to a representative sample of young people in World Vision’s project area. The chart below shows the average score for each of the 12 statements.

The grit scale (average score on the above statements) is quite a good predictor of labor force participation. A person is considered outside the labor force if they do not have a job and are not interested in one, looking for one, or able to start one. A person is considered in the labor force if they are employed or are looking for a job, can start one, and are interested in one. The chances of whether someone will be in the labor force increase significantly as an individual’s grit increases. This pattern holds when adjusting for other factors including age, sex, parental education level, whether the person was displaced by a conflict, family size, and municipality. The chart below shows the probability of participation in the labor force adjusted for each these factors. It suggests that all else equal, if a person moved from the lowest score observed (1.4) to the highest (5), their chances of participating in the labor force would increase from 47% to 82%, a 35 percentage point increase in the probability of labor force participation.

The pattern is also quite consistent when looking across different demographic groups, with the pattern holding for women and men, people of different ages, from different socio-economic backgrounds, affected and not by the conflicts in the country, from large and small families and in the different municipalities the survey was carried out in.

The above data may suggest that grit may help in getting a job in Georgia, a positive story given that people often think connections are more important than hard work for finding a job. Given this, it also suggests that the grit scale works in Georgia as in other contexts, giving some amount of validity to it outside the United States where it has been used extensively.

The views presented in the above blog post are the author’s alone and do not represent the views of World Vision or the European Union.


Monday, May 13, 2019

Pessimism about Georgia’s direction hides room for optimism

[Note: This article was co-published with OC-Media. The article was written by Koba Turmanidze, the Director of CRRC-Georgia. The views expressed in this article represent the author’s alone and do not necessarily represent the views of the National Democratic Institute or any other entity.]


While a large number of Georgians think the country is going in the wrong direction, the fact that they are judging the country’s performance based on issues rather than political partisanship alone is a good sign.

A quick and simple look at where people think the country is headed is not very hopeful in Georgia: NDI/CRRC survey data show that at least one in three adults believe that the country has been going in the wrong direction for the past five years, and for most of that time, more people have reported the country was going in the wrong direction than the right one.

While a first look suggests a less than rosy picture, the data do hide some positive news. People, at least in part, appear to judge direction based on policy performance rather than only whether their preferred party is in power, something that has inhibited the development of a stable political and party system in Georgia.

Looking at demographic factors that might influence assessments of the country’s direction, including age, gender, education, employment, household economic status, and household size suggests demographics explain relatively little in terms of attitudes towards the direction of the country. Only tertiary education is associated with having a more negative attitude towards the direction of the country among these variables.

Yet, a statistical analysis that includes people’s assessments of specific policies and party preferences shows a strong link with how people perceive the direction of the country. People who negatively assess a specific issue are two to three times more likely to assess the country’s general direction negatively. Of 16 issues asked about on the survey, the only exception was inflation, where a negative assessment influences perceptions of the country’s general direction relatively little, all else equal.


Surely, some issues are more important for people than others, jobs being at the top of the list in Georgia. In contrast, freedom of speech was close to the bottom at the time of the survey, with only 2% naming it as an issue of national importance in the same survey wave.

Yet, no matter the relative importance of the issue, the relationship described above still holds. The chart below illustrates the point. A person with a negative assessment of the country’s direction is 25 percentage points more likely to say that the situation regarding jobs is going in the wrong direction. Likewise, people who say that the situation regarding freedom of speech is going in the wrong direction are 32 percentage points more likely to assess the country’s direction negatively.


Attitudes towards political parties are also associated with assessments of the country’s general direction. On the survey, people were asked whether there was a party they would never vote for, a question used to measure negative partisanship.

As one would expect, a negative attitude towards Georgian Dream, the ruling party, is positively associated with a negative assessment of Georgia’s general direction, while a negative attitude towards the United National Movement is associated with more positive assessments. This holds for both the direction of the country as well as individual policies in most cases.


While people’s partisanship matters, so do their assessments of particular issues. Both predict whether or not someone thinks the country is headed in the right or wrong direction, controlling for the other factors.

The chart showing assessments regarding each of the 16 issues by negative attitudes towards the two largest parties illustrates the point. Whether people dislike Georgian Dream or the United National Movement, a negative assessment of a specific issue is associated with a negative assessment of the country’s direction. The same observation holds for people who do not hold a negative predisposition towards any political party.

This matters. Citizens are not looking at specific and general issues through narrow partisan lenses alone. Instead, the data suggest assessments are at least partly independent from party labels, which provides parties with the opportunity to campaign on issues instead of merely blaming each other for their failures and attempting to cultivate followings around charismatic leaders.   

Note: The above analysis is based on a series of logistic regressions, where the dependent variable is a negative assessment of Georgia’s general direction, key independent variables are a negative assessment on each of 16 specific policy issues as well as attitudes towards political parties. In addition, all models have demographic control variables including, gender, age, settlement type, education, employment status, household size, and household’s economic status. Replication code of the full analysis is available here

Tuesday, May 07, 2019

Men report doing more at home than they likely do in Armenia and Georgia

[Note: This article first appeared in OC-Media.]

In Armenia and Georgia, traditional gender roles continue to define the division of labour within families. Although a few tasks are within men’s domain and a few others are more or less equally shared, for the most part, women hold the primary responsibility for household duties.

However, men and women also have different perceptions of how much work each are doing: data from a UN Women survey on women’s economic activity and engagement in the informal labour market suggests that men tend to overestimate their own household contributions relative to women’s.

Most Armenians and Georgians see household tasks as divided according to standard gender roles. As shown below, when asked who in their household was mainly responsible for certain tasks, people said that cleaning, cooking, and laundry are often done by women. More Georgians than Armenians said that men share the responsibility of childcare with women: 28% of Georgians said that both male and female household member are responsible for childcare, compared to 12% of Armenians.


Men tend to have fewer household duties. Among the long list of household tasks asked about, ‘Repairing things around the household’ is the only activity for which men are primarily responsible. Grocery shopping is the only household activity with a distribution perceived to be near-equal. The only other task where there is a relatively higher number of shared responsibilities is related to ‘Taking care of other family members’, although the task is still mostly performed by women. These percentages are similar across Armenia and Georgia.



However, men and women also have different perceptions of how household labour is split— and the data suggests that men tend to overestimate the relative share of their contributions. Across a wide range of household tasks, men are more likely than women to report that duties are shared equally. Women, on the other hand, are more likely to report that tasks are their responsibility. This gap is larger in Georgia, where men report more involvement in household labour than in Armenia.

The perception gap is more striking for the activities in which men reported higher levels of equal involvement. For example, as we see below, 20% of Armenian men and 40% of Georgian men said that childcare duties were shared equally, while only 7% of Armenian women and 18% of Georgian women agreed. Regarding grocery shopping, 57% of Armenian men and 57% of Georgian men said that men and women were equally responsible, while only 39% of Armenian women and 42% of Georgian women agreed.


This is not unique to Armenia and Georgia: it’s consistent with trends from other contexts showing that even when both parents work outside the home and aspire towards an equal division of household labour, in practice, women usually still end up doing more. Surveys from the US show that although men are doing more in the house than ever before, they still do not do as much housework or childcare as their partners. Despite this, men are more likely to say that duties are shared equally.

Indeed, in Armenia and Georgia, when asked about the actual time people spent on tasks, women were likely to report higher average hours for most tasks. While men and women reported roughly equal time spent grocery shopping, and men spent more time fixing things, the amount of time required for these tasks was much less than women reported spending on childcare, cooking, or cleaning.


So how do men and women feel about this? Despite these differences, most people in Armenia and Georgia said they expressed satisfaction with their household labour division.  Only 6% of Armenian men, 10% of Armenian women, 5% of Georgian men, and 9% of Georgian women said that they were either ‘dissatisfied’ or ‘very dissatisfied’ with the breakdown of labour in their household.
Gender stereotypes appear deeply rooted in labour distributions in families in both countries. Women bear primary responsibilities for most household activities, while ‘repairing things around the house’ is the only predominantly male activity. Even when men think they are sharing household duties, in practice, women are still likely to be doing more.

This article was written by Meagan Neal, an International Fellow at CRRC-Georgia, and Kristina Vacharadze, CRRC-Georgia’s Programmes Director.

The data used in this article are available at CRRC-Georgia’s Online Data Analysis tool

The views presented in this article are the views of the authors alone and do not represent the views of UN Women.