Monday, September 26, 2016

Trends in the Data: Declining Trust and Rising Ambivalence towards the Media in Georgia

CRRC has written before about the ambivalent attitude of the population of Georgia towards journalists. Based on CRRC’s Caucasus Barometer (CB) survey data, this post explores the population’s trust in the media over time, showing that it has been declining steadily since 2008, while ambivalence, demonstrated by the finding that people have difficulty stating their opinion and opt instead for either/or options, has been increasing. 

Between 2008 and 2015, reported trust in the media declined by 28 percentage points in Georgia. The biggest drops are between 2008 and 2009 and between 2011 and 2012. Interestingly, over the same period, the reported level of distrust in the media has remained rather steady. Ambivalence, however, is rising. The share of the population responding that they “neither trust nor distrust” the media climbed from 28% in 2008 to 54% in 2015.






Note: A 5-point scale was used during the survey. For this chart, answer options “Fully trust” and “Trust” have been combined into ’Trust’ and answer options “Fully distrust” and “Distrust” have been combined into ’Distrust’. Caucasus Barometer was not carried out in 2014.

The decline in trust and rise in ambivalence towards the media is consistent with responses to other CB questions on the media. Over time, the population’s positive assessment of how well TV journalists inform the population about what is going on in the country has also declined, while their ambivalence has risen. Since both trends are steady, this change seems to be less of a question of a reaction to specific events and more of a general shift. Between 2009 and 2015, positive assessments of how well TV journalists inform people dropped by 14 percentage points while ambivalence increased by 12 percentage points. As in the answers about general trust in the media, reported negative assessments remain stable.




Note: A 5-point scale was used during the survey. For this chart, answer options “Very well” and “Quite well” have been combined into option ‘Well’ and answer options “Very poorly” and “Quite poorly” have been combined into ‘Poorly’. 

Similar patterns can be discerned with regards to whether the Georgian population thinks that TV journalists serve their interests. Since 2009, the share of the population who reported believing that TV journalists, overall, serve the interests of people like them decreased by 13 percentage points, while ambivalence increased by 15 percentage points. 



Note: A 5-point scale was used during the survey. For this chart, answer options “Completely agree” and “Somewhat agree” have been combined into option ’Agree’ and answer options “Completely disagree” and “Somewhat disagree” have been combined into ’Disagree’. This question was not asked in 2013. 

The population of Georgia’s trust in the media has been steadily declining since 2008. Interestingly, this decline coincides with an increase in ambivalent attitudes rather than distrust. The same is reflected in assessments of how well TV journalists keep the public informed and how well they represent the interests of “regular” people. In all cases, positive assessments have decreased, while ambivalence has increased. If the trends marked here are indeed general shifts in attitudes towards the media, as the data for available years suggests, this has the potential to point to long-term changes that are less attached to specific political events or media scandals and may indicate avenues for further research on public opinion in Georgia about the media. 

The datasets used in this blog post and related documentation are available at our online data analysis platform. 

Monday, September 19, 2016

Employment and income in Georgia: Differences by educational attainment

According to the data of the National Statistics Office of Georgia for 2005-2016, there are approximately 100,000 students in Georgian tertiary educational institutions. Around the world, education generally contributes to increased individual income, and Georgia would not be expected to be an exception in this regard. Still, the role of tertiary education in the professional lives of the population of Georgia has not been studied thoroughly. Based on CRRC’s 2015 Caucasus Barometer survey, this blog post looks at the share of the population that has completed tertiary education, what share of those are employed and in what positions, how much their personal income is, and how the employment situation of those with tertiary education differs from the situation of those who did not obtain a degree.

The answers to the following questions, which used show cards are analyzed in this blog post:
  • What is the highest level of education you have achieved to date? 
    • show card listing levels of education was used.
  • Which of the following best describes your situation?
    • A show card with the following answer options was used:
      • Retired and not working;
      • Student and not working;
      • Housewife and not working;
      • Unemployed;
      • Working either part-time or full time (even if the respondent is retired / is a student), including seasonal work;
      • Self-employed (even if the respondent is retired / is a student), including seasonal work;
      • Self-employed (even if the respondent is retired / is a student), including seasonal work;
      • Other.
  • Which of the following best describes the job you do?
    • A show card listing a hierarchy of job types was used.
  • Speaking about your personal monetary income last month, after all taxes are paid, to which of the following groups do you belong?
    • A show card with income groups was used.
Thirty percent of Georgia’s population reports having completed tertiary education (Bachelor’s, Master’s, Specialist’s or post-graduate degree). As the chart below shows, 29% of those without tertiary education report being employed compared to 49% of those with tertiary education.


Note: Answer options to the question “What is the highest level of education you have achieved to date?” were recoded in the following way: “No primary education”, “Primary education (either complete or incomplete)”, “Incomplete secondary education”, “Completed secondary education”, “Secondary technical education” and “Incomplete higher education”  were combined into “Do not have tertiary education”. Answer options “Completed higher education” and “Post-graduate degree” were combined into “Have tertiary education”.

Answer options to the question “Which of the following best describes your situation?” were recoded in the following way: “Working either part-time or full time (even if retired / a student), including seasonal work”, “Self-employed (even if retired / a student), including seasonal work” were grouped as “Employed”. Those who answered “Disabled and unable to work” and “Other” (2%) were excluded from the analysis. Answer options: “Retired and not working", "Student and not working", "Housewife and not working", and "Unemployed" were grouped as “Unemployed”. Within this group, those who answered “Yes” to the question “Are you currently interested in a job, or not?” were grouped as “Unemployed who are interested in a job”, while those who answered “No” were grouped as “Unemployed who are not interested in a job”.  

Answers “Don’t know” and "Refuse to answer” to either of these questions were also excluded from the analysis. Overall, 4% of cases were excluded. 

As for job positions, most of those with tertiary education who were employed at the time of the survey (28%) were employed as professionals (in the fields of science, healthcare, education, business, law, culture, etc.). On the other hand, most of those without tertiary education who were employed at the time of the survey (18%), reported working in the service sector (e.g., as salespersons, including personal care workers, e.g. baby sitters). 

The higher the income group, the higher is the share of those with tertiary education in it. For example, almost there are almost 2.5 times as many people with tertiary education among those who earned above GEL 600 the month before the survey, compared to those without tertiary education. A Mann-Whitney test shows that the difference between these groups is statistically significant. 


Note: Answer options to the question “Speaking about your personal monetary income last month, after all taxes are paid, to which of the following groups do you belong?” were recoded in the following way: options “GEL 601 to GEL 1000”, “GEL 1001 to GEL 2000”, “GEL 2001 to GEL 3000” and “More than GEL 3000” were grouped as “More than GEL 600”. Answer options “Up to GEL 120” and “GEL 121 to GEL 240” were grouped as “Up to GEL 240”. Those who answered “0”, “Don’t know”, and “Refuse to answer” were excluded from the analysis (36% of cases).

The findings presented in this blog post show that, like in many other countries, tertiary education plays a positive role for employment prospects in Georgia. People with tertiary education are more likely to be employed compared to those who do not have tertiary education. The largest group of those with tertiary education is employed as professionals, while those without tertiary education are most frequently employed as service workers. Importantly, the income of those with tertiary education tends to be higher. In all cases, the differences between those with and without tertiary education are statistically significant.

For more information about the impact of education, see CRRC’s earlier blog posts including Educated parents, educated children? And Connections or education? On the most important factors for getting a good job in Georgia. For more data, check out our Online Data Analysis tool.

Monday, September 12, 2016

Trends in the Data: Changes in the level of trust in social and political institutions in Armenia

According to an earlier CRRC blog post, which looked at the changes in the level of trust in social and political institutions in Georgia from 2011 to 2015, trust in a fair number of institutions in Georgia declined. This post provides a comparable review of the situation in Armenia, using CRRC’s Caucasus Barometer (CB) survey data.

The level of trust in most political institutions CB asked about has declined in Armenia since 2011. The largest decline can be observed in respect to the President. Trust dropped from 36% in 2011 to 16% in 2015. Trust in executive government and parliament also declined between 2011 and 2013, and has stabilized since at a rather low level.

Note: The charts in this blog post only show the share of those who report trusting the respective institution. Answer options “Fully trust” and “Rather trust” were combined.

The survey results also show a slight decline in trust in courts between 2011 and 2015. Trust in the police, educational system and healthcare system remained largely unchanged, while trust in the army increased.




In sum, of the institutions CB asked about, the largest drop in the level of trust is observed was in the President, while trust in the army increased in Armenia. The levels of trust in executive government, parliament, and courts in Armenia have slightly declined since 2011, while the levels of trust in the healthcare system, police and educational system have not changed.

To learn more about trust in institutions in the South Caucasus, take a look at the data using our Online Data Analysis tool.

Friday, September 02, 2016

Trends in the Data: Declining trust in the banks in Georgia

The last few years have been turbulent for Georgia’s national currency, the Lari (GEL), the value of which started to decline in November 2014. While in October 2014 one US dollar traded for GEL 1.75, since February 2015 to date, the exchange rate has fluctuated between GEL 2 and 2.5 per dollar. Needless to say, the depreciation of the Lari has been widely covered by the media, and although it had numerous causes, a number of organizations and people were blamed for the devaluation. With this background in mind, this blog post looks at how reported trust in banks has changed in recent years in Georgia, using CRRC’s Caucasus Barometer (CB) survey data.

In 2015, for the first time since CB started asking the population about their trust in banks, more people in Georgia reported distrusting than trusting them. The decline in trust, however, started well before the GEL began to depreciate. While 27% reported trusting banks in October 2015, 53% did in October 2008.


Note: The original five-point scale was recoded into a three-point scale for this chart. Answer options “Fully trust” and “Trust” were combined into the category ‘Trust,’, while “Fully distrust” and “Distrust” were combined into ‘Distrust.’ “Neither trust nor distrust” was not recoded. The Caucasus Barometer survey was not conducted in 2014.

As is generally the case with trust in social and political institutions in Georgia, the population of rural settlements report less distrust in banks than residents of urban settlements. Nonetheless, since 2008, distrust in the banks in rural settlements has nearly tripled, from 11% in 2008 to 30% in 2015. In the capital, distrust has almost doubled during the same period.
 

Although there has been a decline in trust in the banks in recent years, this decline started before the devaluation of the Lari began in 2014. While the devaluation likely contributed to the decline in trust, the fact that trust began declining earlier shows that there is more to the story than the devaluation.
Given that the banking system, and trust in it, is crucial to the effective functioning of a country’s economy, the government of Georgia and banks themselves should consider efforts aimed at building trust in the banking sector.

What factors are at play in declining trust in the banks in Georgia? Join the conversation on the CRRC-Georgia Facebook page here, and to explore more data on Georgia and the South Caucasus, visit our online data analysis tool (ODA).

Monday, August 29, 2016

Trends in the data: A majority of the population of Georgia now uses the internet

Internet use is on the rise worldwide, and while internet penetration is increasing the world over, in some countries still relatively small shares of the population use it. While only about a third of the population of Georgia reported using the internet at least occasionally in 2009, today, slightly over half of the population is online. This blog post looks at the trends in internet usage in Georgia from 2009 to 2015 by age and settlement type, using the CRRC’s Caucasus Barometer survey (CB) data.

Until 2012, a larger share of the population had reported never using the internet or not knowing what it was than reported using it. In 2013, roughly equal shares reported using and not using the internet. On CB 2015, a majority (57%) of the population of Georgia reported using the internet at least occasionally.

Note: The original question asked: “How often do you use the internet?” For this blog post, answer options “Every day”, “At least once a week”, “At least once a month”, and “Less often” were combined into ‘Yes’. Answer options “Never” and “I don’t know what the internet is” were combined into ‘No’. 

Throughout this period, as one might expect, a larger share of the younger generation has used the internet than the older generation, with 87% of 18-35 year olds reporting using the internet at least occasionally in 2015, compared to only 19% of people aged 56 and older. Interestingly, the largest increase is among those between the ages of 36 and 55, with only 24% of this age group using the internet in 2009 compared with 63% in 2015.

Note: The chart above presents only the shares of those who reported using the internet at least occasionally. 

As is well known, internet usage is lowest in Georgia’s rural settlements. In 2015, slightly less than half of the rural population (43%) reported using the internet. In contrast, roughly seven in ten residents of urban settlements use the internet. Urban settlements outside the capital have seen the largest increase in internet use between 2009 and 2015, with the share of internet users increasing more than 2.5 times in this period, while the share of internet users has more than doubled in rural settlements.

Note: The chart above presents only the shares of those who reported using the internet at least occasionally. 

Currently, a majority of the population of Georgia use the internet at least occasionally. As one might expect, more young people use the internet than older people, and they use it more frequently. While slightly less than half of the rural population uses the internet, this share is steadily increasing. If this growth continues, a majority of the rural population of the country will soon be online as well.

To explore the data yourself, try our online data analysis tool.

Thursday, August 25, 2016

Making Votes Count: Statistical Anomalies in Election Statistics

[Note: In order to help monitor the fidelity of the October 2016 parliamentary election results, CRRC-Georgia will carry out quantitative analysis of election-related statistics using methods from the field of election forensics within the auspices of the Detecting Election Fraud through Data Analysis (DEFDA) project. The Project is funded by the Embassy of the United States of America in Georgia, however, none of the views expressed in the following blog post represent the views of the US Embassy in Georgia or any related US Government entity.]

On Friday, August 19th, CRRC-Georgia presented and published a pre-analysis report for the Detecting Election Fraud through Data Analysis project, which contained analysis of the new electoral boundaries set up following the 2015 constitutional court ruling that the previous boundaries were unconstitutional.

The report also demonstrated how the methods of statistical analysis that CRRC-Georgia will use to monitor the 2016 elections work in practice. To do so, we used precinct level data from the 2012 party list elections. Specifically, CRRC-Georgia carried out two types of statistical analyses:

  • Logical checks of official election returns, which test whether there were data entry errors when the vote was being recorded and collated; 
  • Tests for statistical anomalies in the official electoral returns, which may suggest electoral malfeasance. 

While Monday’s blog shows the logical checks that CRRC will apply to the final CEC vote records, today we discuss the tests used to identify statistical anomalies in vote counts.

Election Forensics: Detecting statistical anomalies in voting data

Direct observation of polling stations is the best method available to ensure the accuracy of the vote, however, election observers cannot be everywhere all the time. Given this fact, the field of election forensics, a subfield of political science, has developed a number of statistical tests to look for statistical anomalies in election returns, which may suggest suspicious election-related activity. Although a number of rather complicated statistics exist, we focus on a number of simpler tests. Specifically, we use tests based on the distribution of the second digit in the number of votes cast, the final digit in the number of votes cast, and the distribution of turnout within an electoral district.

Second digit tests are based on Benford’s law. Benford’s law provides the expected probability of the first digit being any digit one through nine in a number with multiple digits. Although one might expect this number to be equally likely to be any number, in fact 1 is more likely than 2, 2 more likely than 3, etc. Using Benford’s Law, accountants test various documents for anomalies that may suggest issues in documents. This law also applies to the second digit in a number, which researchers have found is more suitable for testing election results. A similar logic is applied to elections as in accounting, and in this blog, we specifically test whether the skew, kurtosis, and the average of the second digit and its distribution follow the expected distribution or not. Instances of non-conformity to Benford’s law may suggest electoral malfeasance.

Besides second digit tests, a number of tests have been proposed for the last digit in vote counts. Here, the expected distribution of digits is much more intuitive, and one expects each digit, zero through nine, to be approximately 10% of the total distribution. Based on this distribution, we test the mean of the last digit and of the mean of the count of zeros and fives in the final digits of votes.

In order to test whether the above noted digit tests in fact indicate potential issues or whether the difference between the observed and expected values was a chance variation, we use a statistical method called bootstrapping. This method lets us to estimate 99% confidence intervals. In the present case, the confidence intervals provide a range within which the result could have fallen by chance. If the range covered does not include the expected value for a given test statistic, we conclude with 99% confidence that the number is different not by chance alone.

Finally, voter turnout is expected to have a relatively normal distribution with a single mode. Based on this expectation, we test whether voter turnout in each electoral district has a single mode or multiple modes using what statisticians refer to as a dip test.

Before reporting the test results, it is worth noting several important caveats when interpreting these tests:

  • Test results are probabilistic, which means that they say the distribution is highly unlikely (would occur 1% of the time in the present case), rather than impossible to occur in the absence of issues. For the tests, we calculated 99% confidence intervals. With 99% confidence intervals and having conducted 444 tests on the 2012 proportional election results, statistically we would expect between four and five tests to be set off in the absence of issues due to chance alone. 
  • The lack of a test being set off does not necessarily mean a problem occurred, but it does suggest the need for further examination; 

In total, 11 districts show statistical anomalies in the test results, and a total of 15 tests report suspicious results. Results are presented in Table 5. In the rows with district names and numbers, the actual test values are reported. In the row below the district name, 99% confidence intervals are reported. Red cells in the table indicate the presence of a statistical anomaly.



Rustavi’s electoral returns set off three statistical tests. Given that we have no reason to expect specific voting patterns in Rustavi compared to other areas in the country that did not set off suspicious tests, this suggests that there may have been electoral malfeasance in Rustavi in 2012. Reviews of election monitoring reports, however, did not suggest electoral malfeasance. This test may be picking up on undetected electoral malfeasance from 2012 in Rustavi. Although unlikely, these three tests could have also been set off by chance.

In Kobuleti, two tests were also set off. In Kobuleti, we would not expect a particularly distinctive voting pattern. Hence, there is a relatively strong reason to believe that electoral malfeasance may have occurred in Kobuleti in the 2012 elections. This contention is supported by election monitoring reports, which reported issues in Kobuleti.

In Bolnisi, two tests were set off. Complaints were filed in Bolnisi on election day, and the test may have been set off by these issues. However, given Bolnisi’s relatively high ethnic minority population and distinctive voting pattern, the tests could have been set off by this rather than malfeasance.

Eight other districts had single positive tests for electoral malfeasance, including Vake, Saburtalo, Kareli, Akhaltsikhe, Adigeni, Vani, Senaki, and Martvili. A review of the OSCE and GYLA election monitoring reports suggest that issues may have occurred in at least half of these districts. Although these positive tests could have occurred by chance alone, the four districts in which a test was set off and observers did not report malfeasance in may also suggest unreported problems in the 2012 elections.

This blog post has described the methods CRRC-Georgia will use to detect statistical anomalies in election returns. For more on the methods CRRC-Georgia will use to monitor the elections, see our pre-analysis report, here, and take a look at Monday’s blog post on logical inconsistencies in election records.

Monday, August 22, 2016

Making Votes Count: Logical Inconsistencies in Voting Records

In order to help monitor the fidelity of the October 2016 parliamentary election results, CRRC-Georgia will carry out quantitative analysis of election-related statistics using methods from the field of election forensics within the auspices of the Detecting Election Fraud through Data Analysis (DEFDA) project. The Project is funded by the Embassy of the United States of America in Georgia, however, none of the views expressed in the following blog posts represent the views of the US Embassy in Georgia or any related US Government entity.

On Friday, August 19th, CRRC-Georgia presented and published a pre-analysis report for the project, which contained analysis of the new electoral boundaries set up following the 2015 constitutional court ruling that the previous boundaries were unconstitutional. The report also demonstrated how the methods of statistical analysis that CRRC-Georgia will use to monitor the 2016 elections work in practice. To do so, we used precinct level data from the 2012 party list elections. Specifically, CRRC-Georgia carried out two types of statistical analyses:

  • Logical checks of official election returns, which test whether there were data entry errors when the vote was being recorded and collated; 
  • Tests for statistical anomalies in the official electoral returns, which may suggest electoral malfeasance. 
While today’s blog shows the logical checks that CRRC will apply to the final CEC vote records, tomorrow we will discuss the tests used to identify statistical anomalies in vote counts.

Logical inconsistencies in voting records
For the 2016 elections we will carry out two types of checks of the logical consistency of votes. Specifically, we will check:
  • Whether there are more or less votes and invalid ballots than signatures recorded on voter rolls;
  • Whether turnout increases over the course of the day.
Voter signatures - Votes recorded - invalid ballots ≠ 0
Taken together, the number of signatures recorded for ballots minus the number of votes recorded minus the number of invalid ballots should equal zero. However, in the 2012 parliamentary proportional list elections this was not the case in approximately 25% of precincts. From the 3,680 precincts which had ten votes or more:
  • 936 precincts had more or less signatures than votes and invalid ballots (25% of all precincts); 
  • Of these, 918 had more signatures registered than votes recorded for a party or ballots registered as invalid combined; 
  • 18 precincts had fewer signatures than votes registered for a party and invalid ballots combined.
These phenomena likely have numerous causes. While some are problematic, others are benign.

To start with the 918 cases of fewer votes registered for a party or invalid ballots than signatures recorded, the severity of the issue varies widely. In order to provide some sense of the gravity of the issue, we have grouped precincts by the number of extra signatures into three categories: unlikely to be problematic (1-9 extra signatures), potentially problematic (10-49 extra signatures), and suspicious (50 or more extra signatures). Table 1 presents the number of precincts that fall into each category:


Unlikely to be problematic Potentially Problematic Suspicious
# of Precincts 816 (89%) 56 (6%) 46 (5%)
Count foreign 0 4 42

Notably, of the 46 suspicious cases, 42 are in foreign precincts. With foreign precincts, we strongly suspect that there was a data entry error as discussed in more depth in our report. Among domestic precincts, there are four suspicious precincts with more than 50 extra signatures. In Marneuli’s 22nd precinct, there were 51 extra signatures. In Khashuri’s 32nd precinct, there were 63 extra signatures. In Gori’s 63rd precinct, there were 71 extra signatures, and in Bolnisi’s 62nd precinct, there were 87 extra signatures.

Potential causes for this situation include voters coming to polling stations, and:
  • Signing the voter list and leaving without voting;
  • Voting only in the majoritarian race rather than in both the proportional and majoritarian races;
  • Additionally, Precinct Electoral Commissions may have inaccurately recorded votes, invalid ballots, and/or signature counts.
In 18 cases, there were less signatures on voter rolls than ballots declared invalid and votes recorded. In 17 of the 18 cases there were 10 votes or less that were without a signature. However, in Gori there were 196. This may stem from a recording error, since there was a very high number of invalid ballots (221), or this may stem from another issue. Generally however, the causes of there being more votes and invalid ballots than signature recorded, the causes are less benign. They include:

  • Precinct electoral commissions may have incorrectly counted or reported vote statistics;
  • Voters were allowed to vote without signing the voter list;
  • Ballot box stuffing occurred.

Declining turnout
Another clear logical inconsistency in the official statistics on the 2012 elections is that the number of votes in several precincts declined between 12PM and 5PM, as well as in one district between 5PM and 8PM. That is to say, according to the official record, fewer people had voted at 5PM, in total, compared to five hours earlier at 12PM in these districts.

District
Saburtalo Nadzaladevi Dmanisi Dmanisi Akhalkalaki Mestia Kobuleti
Precinct 63 44 23 30 48 25 14
Votes between 12PM and 5PM -1 -159 -19 -58 -40 -43 -210

This is likely to be caused by a reporting error, with precinct officials recording the number of votes between these hours rather than the total number of votes at 5PM.

Conclusions
While each of the above logical inconsistencies in recording the vote is clearly an issue, which could imply malfeasance, we strongly suspect that the vast majority of cases described above stem from recording and data entry errors. While, we do not suspect malfeasance in any particular case, and do not believe that recording issues affected the outcome of the 2012 elections, the illogical recording of the vote is a serious issue.

In Georgia, elections and the outcomes of elections are regularly contested, with accusations of all sorts following the results. If Georgian voters see that the voting records have logical inconsistencies in them, this could undermine citizens’ confidence in the accuracy of the vote, and thus the legitimacy of election results.

Based on this, we recommend that the Central Election Commission, District Election Commissions, and Precinct Election Commissions check for logical inconsistencies in election protocols on election day and explain logical inconsistencies in a public and transparent manner if they do occur. Particular emphasis in trainings should be placed on how to fill out voter protocols.

In Thursday’s blog, we show how we will carry out tests for electoral malfeasance in the 2016 elections using tests from the field of election forensics. In the meantime, check out our full report or this visualization of the issues which Jumpstart Georgia created.