SMS Survey | First Insights

So! Our SMS project worked quite well. Critical to its success was the systematic error control early in the day. Our interviewers still made a fair number of mistakes in the early morning. It was the first time we introduced this system, and transferring the number correctly to SMS requires significant attention to detail.

Whenever the system flagged mistakes, we called the interviewer to check, and implicitly to remind them to get it right the next time. The entire CRRC team took turns, with two colleagues in each shift, in addition to the other people in our E-Day room (the cheerful morning group pictured above). Note that we chose codes that made a transposition error unlikely. Since "Yes" was 1, and "No" was 3, you'd have to go across the keyboard to get that wrong. 

The chart shows the error rates. In the morning, we quickly manage to reduce the errors by almost half in the first hour. In the afternoon, some exhaustion sets in, but the interviewers recover in the early evening. In the last hour, however, error rates almost quadruple within an hour -- our interviewers are well and truly tired. 

More detail on this to follow. We will also enter the paper questionnaires, and then compare with the SMS results, to see whether there actually were any transposition errors. For us, this was an exciting day. 

Testing Mobile Innovation in our Surveys

In running an election-day survey (not an exit poll, which we are not so enthused about), we have decided to attempt something new: we are now aggregating the information via SMS. This gives us the information in real time, and the data will be available for immediate analysis the moment the last SMS has been received. On the image below you see how this looks on our screen. Interviewers send in a code response where letters (A, D, G, and so on) signify the question number, and the numbers are the response chosen. So the letter M, for example, stands for the question "Did you, or anyone from your family, have any problems with the voter’s list?", and "3" stands for "no".

To be sure, this only works with short questionnaires, and has forced some compromises in the answer options, which we have to keep simple. However, it is an interesting step forward. While there are solutions with Personal Digital Assistants, these are expensive and would make our interviewers walk around with expensive, possible distracting gadgets. So SMS seems a good alternative for now. Implementing this project ended up being a lot of work: creating a Virtual Private Network connection with the main mobile providers (Magti and Geocell both were helpful), programming the software to disaggregate messages, writing manuals, training interviewers, testing whether it all works, building redudancies so that it becomes a robust system, and creating a system for checking errors. Errors are flagged automatically in red (see below), and we call back interviewers to correct. 

We are also trying this SMS technology with the election monitors of ISFED, since this offers a rapid way of aggregating their results. At this point this is run as a pilot, to test the technology, and how it is adopted by monitors. Parts of the software have been provided by the National Democratic Institute, and the Open Society Foundation Georgia has generously funded this effort. A lot of effort has been invested by the entire team, but especially by Irakli Naskidashvili, Tbilisi's IT wizard, and Jonne Catshoek, our Crowdsourcing Project Manager.

This will all happen today, and we will let you know how it worked, and where we want to take these applications in the future. One of the great things at CRRC is that we have this chance to goof around and try new things. 

Election Day Portal

To track what is going on during election day, Georgia's leading monitoring organizations, the International Society for Fair Elections and Democracy (ISFED), the Georgian Young Lawyers Association (GYLA) and Transparency International (TI) have created a joint portal, VoteGeorgia.ge. 

This will join the feeds from the three organizations, while also giving you a map with region-specific information. The website has been designed by TI. NDI provided critical coordination, as well as access to survey results. CRRC is providing the maps for the effort. We work with GeoCommons to provide the data on the maps. Below a snapshot of a pre-election complaint. 

Note that the maps take time to load. They are not as fast and snazzy as Flash-based maps would be, since they get populated by data that is continuously updated, during the election day and after. Here the technology is still catching up -- we are also using this as an exercise to learn how best to make mapped information available over the Internet. 


We know there are some snags -- it does not work too well with the Safari browser, for example. Further suggestions gratefully received in the comments. This is a pilot, and we want to use this opportunity to get everything right in the future. 


On CRRC's side, David Sichinava, our GIS and Database Analyst, and Jonne Catshoek, our Crowdsourcing Project Manager, were the ones who made this happen. This work has been supported generously  by the Open Society Foundation Georgia. 

Levels of trust in the banks in Georgia: Changes over the past two years

Banking is one of the fastest-growing sectors of the Georgian economy, a point which was underlined in a 2009 report from the Ministry of Economic Development of Georgia. But does this development mean that society views banks as trustworthy partners for households (HH) in Georgia?

In fact, from 2008 to 2009, the overall level of trust in banks has decreased in Georgia, especially for the HHs who say they have savings and for those who say they have debts. This could in part be due to the global financial crisis which, according to the European Bank for Reconstruction and Development’s (EBRD) 2010 country strategy report, significantly affected the Georgian economy. The crisis revealed the financial sector’s weaknesses around the world and led to widespread doubt concerning the reliability of banks.

Households & Savings

For the small number of HHs who say they have savings (9 percent in 2008, 6 percent in 2009), the level of trust in banks has significantly decreased. According to the 2008 Caucasus Barometer (CB, previously referred to as the “Data Initiative”), combining the “fully trust” and “somewhat trust” categories shows that 60 percent of HHs with savings said they trusted banks. In 2009, however, this figure fell to 49 percent. As a place to put one’s money and keep it safe, apparently, fewer people view banks in a positive light.

Even for HHs without savings – the majority of the respondents – the level of trust in banks has fallen: in 2008, 53 percent of them had said that they trusted banks, whereas only 42 percent said the same in 2009 (see Figure 1). At the same time, the number of those saying specifically they distrust banks remained the same, hinting at a high degree of uncertainty with regard to banks among the population.



Households & Debts

The level of trust in banks among HHs who reported that they have debts saw an even larger drop. Although these could be debts either to banks or to private persons, without interest, this drop could be linked to the fact that it has become increasingly difficult for HHs to take out loans to help alleviate any debts they have. Overall, 43 percent (2008) and 42 percent (2009) of the HHs claimed to be indebted. Of these HHs, 59 percent said that they trusted banks in 2008, though only 45 percent claimed the same in 2009.

On the other hand, the level of trust remained broadly similar in those HHs who say they do not have debts. Forty-nine percent of them had said that they trusted banks in 2008, while 42 percent said the same in 2009.




Conclusion

The 2008 Caucasus Barometer was carried out from the second half of October to the middle of November, and these figures offer a snapshot of how the global financial crisis may have taken its toll on HHs’ trust in banks in Georgia. Still, there are certainly other factors which play a role, and further research and commentary are needed. We invite you to access the 2008 CB dataset to make your own comparisons here.

Caucasus Barometer | A New Name for the CRRC's Data Initiative

The CRRC’s annual Data Initiative Survey will be renamed into the Caucasus Barometer starting from 2010. At CRRC, we think that the new name better reflects the essence of the survey and is more understandable for the general public and the journalists.

The Data Initiative was first launched in 2004. Since 2007, a representative sample of approximately 2,000 respondents is interviewed annually in each of the counties. They answer core questions about household composition, social and economic situation of households, employment status, assessments of social and political situation in the countries, as well as respondents’ perceptions about direction of life. In addition, we include questions about media, health, crime, and other topical issues.

The change of the name, however, will not cause any changes in the way the survey is carried out – it is still an annual survey conducted every fall in all countries of the South Caucasus, employing the same methodology and the same survey instrument. Its major goal is to get reliable longitudinal empirical data to understand various aspects of the processes of social transformation in Armenia, Azerbaijan, and Georgia. We are committed to ensure the highest possible scientific quality through all the steps of survey implementation.

The data and the survey documentation are open to all interested researchers and represent a unique tool for further quantitative analysis. You can find more information about the Data Initiative/Caucasus Barometer on our website.

The Level of Trust in Government Institutions in Georgia: The Dynamics of the Past Three Years

During the last two decades, Georgia has created new government institutions designed to serve as the tools and safeguards of democracy. But do Georgians believe that these institutions live up to their mission statements? How much do Georgians trust government institutions, and which factors influence the public’s attitudes toward them?

A look at the data from the Data Initiative (DI) over the last three years (2007–2009) shows how the level of trust in government institutions may be related to the political crises and external threats to the country, as well as to the personalities representing these institutions.

For example, the public’s trust in President Saakashvili has experienced a significant increase, especially after the 2008 August war. Combining the “fully trust” and “somewhat trust” categories shows that the percentage of the respondents who trusted the President rose from 32 percent in 2007 to 51 percent in 2008 and remained nearly the same in 2009. Moreover, combining the “somewhat distrust” and “fully distrust” categories reveals that the number of those who distrusted him dropped from 37 percent in 2007 to 18 percent in 2009 (see Figure 1). Such changes could perhaps be linked, firstly, to the events that led to the November 2007 political crisis and, secondly, to the 2008 war and the rise of external threats to the country in the subsequent years. (The DI surveys were conducted in a period of around two weeks, between September and November of the respective years.)

Figure 1: Trust in the President.

Second, the level of trust in the Ombudsman was the highest when this institution was headed by a very popular personality, Sozar Subari (see, for example, Bahrampour, “Georgia’s Counterweight to Power,” Washington Post, July 24, 2009). The level of trust in the Ombudsman rose from 36 percent in 2007 to 58 percent in 2008, while the percentage of those who distrusted this institution fell to seven percent.

When Subari was replaced in September 2009, however, the public’s trust in the Ombudsman’s institution took a reverse turn, dropping by almost a third. While the level of distrust in this institution remained the same, the number of the respondents who trusted the Ombudsman dropped from 58 percent to 40 percent (see Figure 2).

Figure 2: Trust in the Ombudsman.

Unlike the President and the Ombudsman, the levels of trust in the Parliament and the executive government (the Prime Minister and ministers) were lower, perhaps attributable to the lower degree to which most MPs and cabinet members are perceived to be politically independent figures. The level of trust in the Parliament, for example, rose significantly from 19 percent in 2007 to 34 percent in 2008 and remained the same in 2009. Meanwhile, the number of the respondents who distrusted the Parliament dropped from 44 percent in 2007 to 26 percent in 2009 (see Figure 3). The figures in each category of trust in the executive government were nearly the same as that of the Parliament.

Figure: 3: Trust in the Parliament.

Overall, these snapshots show that the public’s trust in government institutions was the lowest in 2007, which coincides with an acute political crisis in Georgia. However, after the 2008 war, the levels of trust grew and, with the exception of the Ombudsman’s institution, remained relatively high compared with the previous years.

You can access the DI dataset here to analyze the public’s trust in other institutions and make comparisons with Armenia and Azerbaijan.