Tracing regional inequalities in the Georgian education system (Part 2)

The first part of this blog post described the regional distribution of 2014 United National Exams (UNE) mean scores in Georgia. Here in the second part, we look at applicants’ gender and location (as explained below) in order to understand how mean 2014 UNE scores differ by these variables. This post also considers the role and quality of teachers in these regional disparities.

A location variable was generated for the present analysis, based on the applicants’ municipality of registration. This variable breaks down the applicants into five groups: those coming from the capital, large cities (Kutaisi, Rustavi, Batumi, Poti), municipalities in Western Georgia (including both urban and rural settlements), municipalities in Eastern Georgia (again, including both urban and rural settlements), and ‘other’ (IDPs and foreign-registered applicants). Although it is not possible to differentiate between urban and rural residents of the municipalities using UNE data, in general, municipalities included in the Eastern and Western Georgia groupings mainly consist of  rural populations.

As descriptive analysis presented in the first part of this blog post showed, applicants from Tbilisi and large cities scored the highest. Applicants coming from ethnic minority municipalities and certain mountainous areas of Western Georgia received the lowest scores. In order to check whether regional differences are systematic or random, a statistical technique called Analysis of Variance (ANOVA) was employed. ANOVA checks whether the mean score of the groups under analysis differ from each other and whether a difference is statistically significant. The ANOVA results show that the main effects of gender, F(1, 26311)= 136.43, p<0.001, and location, F(4, 26311)= 396.36, p<0.001 are significant factors in applicants’ exam scores, while the interaction of these two variables is not significant, F(4, 26311)=1.67, p=0.1531. Post-hoc analysis demonstrates that there are no significant differences between the scores of applicants from the municipalities of Western and Eastern Georgia.

As shown in the chart above, in all locations, female applicants had higher mean scores than males. Applicants from Tbilisi – both males and females – were the most successful. Residents of large urban areas performed better than applicants from predominantly rural municipalities. Finally, applicants from the ‘other’ group scored higher than those from predominantly rural municipalities.

Although, in general, female applicants scored better than males, the difference was not vast. Importantly, there are significant regional gaps between applicants, which are clearly revealed by UNE scores. A number of reasons likely contribute to this disparity.

To start, Georgia is characterized by endemic regional inequality, including uneven quality of education in the country. There is an especially large gap between large urban areas and predominantly rural municipalities. The lack of quality education and good teachers in rural areas is obvious from the results of international tests taken by schoolchildren (TIMSS, PISA, PIRLS). Although, in these tests, Georgian pupils, overall, score around or above world averages, the picture is bleaker when looking at the scores by settlement type.

In Georgia, there is generally a lack of good teachers – nationally, over 90% of teachers of certain disciplines failed their certification exams in 2013. Still, the low quality of teachers in rural areas is more pronounced. If we look at the statistics for pupils per certified teachers (i.e. teachers who have passed special exams and hence are considered better performers compared with their peers), we see that this number is highest in ethnic minority municipalities – that is, there are fewer certified teachers with more students. In Marneuli, Ninotsminda and Akhalkalaki municipalities, there are one hundred or more pupils per certified teacher, while in Sachkhere municipality in Imereti the respective number is 25.

As school education, which should be instrumental in preparing pupils for university admission exams, appears to be inadequate, university applicants (and their parents) often hire private tutors rather than attend school in the final (11th and 12th) grades. Considering that private tutors are in many cases active teachers, the availability of quality tutors in rural areas is also lower, while better private tutors are found in the capital and large urban areas. This factor also contributes to regional disparities in UNE scores.

This series of blog posts explored the results of 2014 Unified Entrance Exams, taking into consideration regional and gender factors. Both descriptive and exploratory analysis shows that there are significant disparities between applicants, especially from the geographic point of view. While UNE was an excellent opportunity for many applicants who would not have had the chance to be accepted to a higher educational institution within the previous corrupt admissions system, certain segments of the population still do not enjoy equal opportunities, not because of the UNE per se, but due to the existing endemic problems that the Georgian secondary education system faces. In spite of its impressive success, the improved university admissions system has not tackled Georgia’s deep-rooted educational inequalities.

Have other insights? Join the conversation on our Facebook page or in the comments section below.

Tracing regional inequalities in the Georgian education system (Part 1)

It has been almost ten years since United National Exams (UNE) for university admissions were introduced in Georgia, and the introduction of UNE has been named by the World Bank as one of the most successful reforms implemented since the Rose Revolution. Previously, university admissions were directly administered by the higher educational institutions, and entrance exams were often the site of highly corrupt practices. UNE ultimately led to the complete elimination of corruption and nepotism from the admissions process.

Corruption aside, fair and exclusively merit-based UNE were expected to give a better chance to applicants from outside Tbilisi, including representatives of ethnic minorities, to enroll in the best educational institutions in Georgia. Some, largely unsystematic evidence, however, suggests that this expectation has not been met. While at present, we do not possess longitudinal data which would enable us to draw comparisons between the situation before and after the reform, we do have data to look at how admitted applicants from different regions of Georgia performed on the 2014 exams. The publicly available 2014 UNE database contains scores for all exam takers (about 26,000 individuals) along with basic demographic data about them, such as date of birth, gender, and municipality where the applicant was registered at the time of exam. It should be noted that the applicants’ place of registration does not necessarily accurately reflect their actual place of residence in Georgia, since no one is obliged by law to live in their place of registration. This is especially true for IDPs, who despite being registered in Abkhazia or South Ossetia, in fact generally reside in areas controlled by Georgia, mainly in Tbilisi (40% of the whole IDP population) and Samegrelo-Zemo Svaneti (32%). However, registration data still makes sense as many people in Georgia, especially the youth graduating secondary schools, generally live in their place of registration.

The map below displays the distribution of UNE scores by applicants’ municipality in 2014. The municipalities on the map below have been assigned a color based on the standard deviation of the applicants’ mean scores. Standard deviation indicates how distant a particular data point, an exam score in this instance, is from the mean. It also “quantifies the amount of variation or dispersion of a set of data values.”  On the map, negative standard deviations indicate values less than the national mean, while positive ones indicate values above the national mean.

  

Looking at the regional distribution of mean exam scores on the map above, a number of patterns can be observed. High and low scores are concentrated territorially and form distinct geographic patterns. Applicants from the capital and large urban areas (Kutaisi, Batumi, Rustavi, Poti) on average have received the highest scores. Another area of concentration of high scores can be observed in Kakheti. The performance of representatives of municipalities from the central-western parts of Georgia was slightly worse (Racha-Lechkhumi, eastern municipalities of Imereti, as well as Khashuri and Gori). Interestingly enough, IDP contestants’ results are also quite high, especially those registered in Sukhumi municipality.

Municipalities with predominantly non-ethnic Georgian populations have the lowest mean exam scores in the country. Applicants from Marneuli, Bolnisi, Akhalkalaki, Ninotsminda and Tsalka municipalities performed the worst on average during the last UNE. Upper Adjara and Svaneti are two other areas with concentrations of low scores.

UNE scores portray a larger problem with the education system in Georgia – regional inequality of access to quality education. Applicants coming from regions and especially ethnic minority applicants are less likely to score high on exams even though they provide everyone with an equal chance. Moreover, provided that advanced knowledge of the state language – Georgian –was necessary to pass the exams, to a certain extent, the UNE discriminated against ethnic minority exam takers, especially those from Samtskhe-Javakheti and Kvemo Kartli, who are often not fluent in Georgian.

Despite the fact that the National Assessment and Examinations Center (NAEC) quickly acknowledged this latter problem and, starting from 2010, offered exams in minorities’ native languages (Armenian, Azerbaijani and Russian), as it is clear from the map above,  the disparity between predominantly Georgian-speaking and ethnic minority municipalities remains. 

This blog post described regional disparities in terms of UNE exam scores in Georgia. In the next blog post, we will investigate whether gender and settlement type impact UNE scores. Despite the above explanations for regional disparities not being exhaustive, they give some food for thought and discussion. Have other insights? Join the conversation on our Facebook page or in the comments section below.

Is the South Caucasus a homogenous region?

In a recent datablog, the Guardian published a map visualizing how the former Soviet countries are doing 20 years after the fall of the Soviet Union. The map compares the 15 former Soviet countries in terms of economic development, demographics and democratic transition. It also divides the countries into five regions: Russia, the Baltic countries, the EU borderlands, Central Asia and the South Caucasus. While the countries in the EU borderland region (Belarus, Moldova and Ukraine) are depicted as going in very different directions, the other four regions are presented as fairly homogenous. This blog will focus only on the South Caucasus region. Is it really a homogenous region as depicted? Survey data from the 2010 Caucasus Barometer (CB) shows that there are striking similarities and differences in the region. This blog will show this by using some questions on gender issues and the perception of the greatest threat to the world in Armenia, Azerbaijan and Georgia as examples.

The Guardian’s datablog compares the performance of the former Soviet countries by making use of statistics from various sources. The result is a map that divides the countries into five geographic regions consisting of countries sharing a specific path of development. The Baltic republics turned west and “never looked back”, Central Asia remains “nostalgic”, and Russia and the Caucasus have “mixed fortunes”. Only Belarus, Moldova, and Ukraine – termed the EU borderlands – develop in different directions, according to the datablog.

The perspective is, of course, a rather general one. However, there are differences with regard to the three countries of the South Caucasus. Georgia is slightly more Western oriented. Azerbaijan also stands out in other respects with population and economic growth since 2000. Nevertheless, in comparison to the other four regions, the Caucasian countries seem to be fairly close to each other in many respects. Live expectancy has risen sharply and infant mortality rates have improved. Furthermore, Armenia and Georgia show a very similar development with respect to GDP per capita.

Source: The Guardian Datablog, 17 August 2011, "End of the USSR: visualising how the former Soviet countries are doing, 20 years on" http://www.guardian.co.uk/news/datablog/2011/aug/17/ussr-soviet-countries-data

Does this picture of a homogenous region persist when we look at the values and perceptions of citizens in the South Caucasus? The 2010 CB shows a mixed picture. For example, a vast majority of citizens in all countries agree that a man should normally be the major breadwinner in a family (83% of Georgians, 84% of Azerbaijanis and 85% of Armenians). The perception that women and man equally share the responsibility for earning the income of a household is supported by 14% of people in Georgia and Armenia and 16% in Azerbaijan. There is also overwhelming disapproval for a woman being the major breadwinner of a family – only 2% of Georgians, 1% of Armenians and 0% of Azerbaijanis support this idea (see below).

While people in the South Caucasus largely agree on the question of who should normally be the major breadwinner, there are considerable differences in perceptions of who the actual breadwinner in the majority of their country's families is. In Azerbaijan and Armenia, 75% and 65% of people, respectively say that the man is the actual breadwinner. Only 6% and 17%, respectively say a woman. The situation in Georgia, however, seems to be very different. 39% of Georgians believe a woman is the actual breadwinner in the majority of Georgian families. This is followed by 36% who say a man is the breadwinner and 20% who think there is an equally shared responsibility among women and men.

Armenians, Azerbaijanis and Georgians also show significant differences in the way they perceive the growing gap between the rich and the poor. In the 2010 CB, they were asked to choose the greatest threat to the world on a list of five dangers. The spread of nuclear weapons was the greatest threat for a plurality of respondents in all three countries. However, in Armenia and Azerbaijan the growing gap between the rich and the poor ranks second with 25% and 21% respectively, while this was the least important of the five dangers to Georgians (7%).

These are only a few examples of similarities and differences among the countries of the South Caucasus. Further similarities (mostly on moral issues) and differences (e.g. western orientation or language preferences) can be examined using our Online Data Analysis tool (ODA).
What similarities and differences do you find most interesting?

Election Maps | Who Did Your Neighbors Vote For?

Our GIS Analyst, Dato Sichinava, is now loading preliminary results onto our election maps. If you want to compare regional distribution of results, you can do so very quickly through the portal. Below you see the votes for the Alliance throughout Georgia. Note that on the portal, you can grab the handle (see the arrow) and shift it back and forth.

We also have the results for the United National Movement, and the Christian Democrats. More stuff will come online soon, and we are about to make the maps bigger, so that you can play better -- check it out here. 


We also have preliminary results for the precinct level in Tbilisi. Note that this is not yet available on the portal. But it shows a fascinating contrast of support. 

Unsurprisingly, Vake is among the strongest supporters of Alasania. Parties should find this data very useful to target their campaigning in the future.

(We would like to thank the National Democratic Institute and the Swedish International Development Agency, Sida, for their support in generating this data, and making it broadly available.)

Election Day Portal

To track what is going on during election day, Georgia's leading monitoring organizations, the International Society for Fair Elections and Democracy (ISFED), the Georgian Young Lawyers Association (GYLA) and Transparency International (TI) have created a joint portal, VoteGeorgia.ge. 

This will join the feeds from the three organizations, while also giving you a map with region-specific information. The website has been designed by TI. NDI provided critical coordination, as well as access to survey results. CRRC is providing the maps for the effort. We work with GeoCommons to provide the data on the maps. Below a snapshot of a pre-election complaint. 

Note that the maps take time to load. They are not as fast and snazzy as Flash-based maps would be, since they get populated by data that is continuously updated, during the election day and after. Here the technology is still catching up -- we are also using this as an exercise to learn how best to make mapped information available over the Internet. 


We know there are some snags -- it does not work too well with the Safari browser, for example. Further suggestions gratefully received in the comments. This is a pilot, and we want to use this opportunity to get everything right in the future. 


On CRRC's side, David Sichinava, our GIS and Database Analyst, and Jonne Catshoek, our Crowdsourcing Project Manager, were the ones who made this happen. This work has been supported generously  by the Open Society Foundation Georgia. 

Nokia's Fictional Georgia Map | Some Way to Go...

For those of you previously following our blog, you will be aware that we have a fascination with maps, and have pointed to the absence of the Caucasus in Google Maps. Recently I tried out the maps feature on my Nokia phone (one of those with a snazzy GPS). I knew that the map would be rough, but I was surprised to find that it had a major road in it that doesn't exist.


As highlighted with the arrow, there is an extension of the A302 that runs right over Tbilisi Sea. No one I have talked to ever had heard about this road being planned, and it wouldn't make much sense either: you would have to build a bridge of nearly 2 km across the waters, when the M27 bypass in the north works perfectly well.

Highlighting this issue to Nokia was complicated. The websites hide themselves behind automated replies; Navtech (the company that supplies the maps and has a feedback mechanism) also gives an automated reply that indicates that map updates are determined by Nokia. Eventually, I did get through on Nokia's new online platform, Ovi.com, where I did get a friendly response and a promise that they would forward the information. Let's see.

Now if that sounds like a lot of effort, it's partially because we think that better maps could make a huge difference. Imagine citizens sending in SMSes if certain public services don't deliver (electricity, water, garbage collection). And imagine if everybody could look at that map. We would instantly know where real problems are.

Where accountability is still developing, mobile devices could provide critical solutions. But for that, the maps would have to get a lot better -- and that's true for Google maps as well.

Caucasus Exceptionalism in Google Maps

While this post is not directly related to social sciences, maps have lots of great social science applications. You can map all kinds of data. One example of this is the Funnel the Money Project.

We keep waiting for Google maps to come out with more data on the Caucasus including basic regional divisions and roads. You see, the Caucasus is a blank slate, except for settlement names.


At first we wondered if this was just a regional thing, so we looked at Turkey. There the data is magnificent. Regional boundaries, and even minor roads are there all the way up to the Georgian border.



Then, we thought it might be a post-Soviet thing. So we looked at Russia. North Ossetia even has good maps similar to Turkey's.

Well, maybe it's just the southern tier of the Former Soviet Union, we thought. However, to our surprise, we saw that Kyrgyzstan (and all of Central Asia) for that matter had road data and Bishkek was even outlined. Sure, it wasn't as good as Turkey or Russia (there were not regional boundaries), but it was significantly better than the Caucasus.

Then we started thinking about the poorest and most war torn of states. Maybe they didn't have road data or other geographic boundaries.

Yet lo and behold, Afghanistan's roads are clearly marked -- though regions aren't.

What about Somalia? Roads again.

Or East Timor? Well, there is only one road, but I am not sure how many there are in the country. Though this map shows more roads.

Finally, we cruised the entire world. The Caucasus are the only three countries without road data.

We know that this data exists for the Caucasus. So, what's up Google?

Mapping Development | WRI's "Funnel the Money"

The World Resources Institute, a global environmental think tank based in Washington DC, is providing maps that allow a visual comparison of data for the countries in the South Caucasus. Called "Funnel the Money", it seeks to chart development within countries, and also track allocation of resources from the central government by providing regional comparisons. The intended users are decision-makers, development specialists and informed citizens, and this interactive tool tries to help them "allocate resources to reduce poverty".

The project remains in draft form, and, we hasten to add, is only as good as the source data. The interface is a little clunky, too.

Still, it is an entertaining way to explore what is going on. You were wondering about the number of Georgian centenarians (those over 100 years old), and how they are distributed across the country?

Here are the male centenarians according to the 2002 census:

And here are the female centenarians. Note that there are many more female centenarians, i.e. that the colors should not be compared to the illustration above.

So apparently there are as many female centenarians in Imereti as there are males in the entire country. And Adjaria has a high proportion of both male and female centenarians.

This is just one of many maps you can extract. What is particularly useful is that even the default shows three maps at the same time, allowing for a good visual comparison.

What would we like to be added? A slightly friendlier interface, an opportunity to compare countries, and maybe a disclaimer regarding data quality. As it is, this still seems to be designed from the data end, rather than from the perspective of the users.

The website notes that they are welcoming comments, a WRI team was just out to visit the Caucasus, so we are hoping for an updated version soon. For now, find the site here.

Caucasian Weather: Asia or Europe?

Is it Europe, or is it Asia? Quite literally, where is it on the map? On a rainy Saturday afternoon we took a look at online weather sites (we were looking for hope online), to check where the Caucasus ended up on alignments that cared little for political affiliations:

• CNN Weather (above) sees the Caucasus as part of Asia. Al-Jazeera (below) agrees, and uses a weather map that mirrors CNN's overall design.

• Accuweather also chooses Asia: in its drop-down menu, Georgia is listed between East Timor and India, while Armenia and Azerbaijan are framed by Afghanistan and Bangladesh. The same view is taken by a number of other sites, including Wetter-Online.
• Wetter.com, a German site, believes that Georgia is part of Europe, but lists Armenia and Azerbaijan as being Asian. Curiously, Georgia is on the Asian map, but in the European list.
  
• Yahoo Weather, powered by the Weather Channel and Weather.com agrees that the Caucasus is Asia, but bizarrely has Turkmenistan in its European listing. Probably an intern was in charge.
  

The BBC Weather Site (above) offers a new perspective: it joins the Caucasus nicely into the Middle East. Visually, at least, this makes a good unit. Black Sea, Caspian, Gulf, Arabic Sea, Red Sea, Mediterranean. Seafarers come to mind. Maybe the geographic unit can also be explained by the BBC's large audience in India and Pakistan -- Asia just needs to be disaggregated as an entity.

There is, of course, a different vision, provided by the Russian weather service:

This world-view is unlikely to be very popular in the Caucasus. The site's forecast, however, is excellent.

• Google Directory listings, by contrast, keep their options open: the Caucasus is listed both in Europe and in Asia. Listing side-steps the tough commitment of a map. This probably is the best compromise for a region in which categories run up against each other.
  

We also have compared forecasts. Sadly, on this, there was consensus: last weekend was rainy.