PISA in Azerbaijan | Take 2 | great maths scores

In a previous post we wrote about the PISA scores of 15-year olds in Azerbaijan. As you may recall, PISA is an international test of competency, primarily focusing on reading, mathematics and science. Azerbaijan deserves particular praise for participating in this challenging international exercise: the results in science were not altogether flattering, but it's better to take part than to stand aside, and it can only be hoped that Georgia and Armenia will also be taking part soon.

At the time of posting, we received some comments that the overall performance was not so bad. Azerbaijani math scores, it was pointed out, were much better. Time, therefore, for another look. Indeed, Azerbaijan performs much better at mathematics. (If you want to see what is being tested, check the PISA sample questions.)

Azerbaijan does better than, say, Argentina, Bulgaria, Mexico, Montenegro, and even Turkey. Conversely, the Baltic states and Russia do better than Azerbaijan. For example, Russia has about 15% reaching Level 4 in Mathematics, and about 6% reaching Level 5 (on a scale from 0 to 6, with 6 denoting highest). Azerbaijan by comparison only has about 7% reaching Level 4 in mathematics, and less than 1% getting to Level 5.

Still, altogether this is highly encouraging news. However, there is one item that is a little hard to explain, and if anyone has any ideas, let us know: according to this OECD data, Azerbaijan has the best basic mathematics training of all participating countries. Only 0.2% do NOT manage to reach the Level 1, which is quite exceptional. In Liechtenstein, for example, a wholesome 4% don't make it to Level 1, in Romania 24%, in Bulgaria nearly 30%, in Brazil even nearly 50%. So with 0.2%, what exactly happened in Azerbaijan? Is it really a case of no-child-left-behind? But what, then, should Switzerland (4.6%), Japan (4%) or Denmark (3.6%) learn from Azerbaijan?

Does anybody know? Did the bad students just not turn up for the test?

Intravenous Drug Users in Tbilisi | Survey Data

As part of a four part series, Save the Children in cooperation with a host of other organizations have released reports from survey data they have collected from Female Sex Workers (FSWs) and Intravenous Drug Users (IDUs). All of the surveys are funded by USAID. This entry reviews the Tbilisi report on IDUs. If you are interested in the other reports, please contact us.

It has been a conundrum for many why the number of HIV infected is not higher in Georgia (Georgia is low prevalence country, less than 5%, as defined by UNAIDS, but data is weak, since surveillance is bad), since it is estimated that there are 250,000-280,000 IDUs in Georgia, which accounts for around 5% of the total population -- an astounding figure. To understand the dynamics of IDUs and its relation to HIV infection other diseases a Behavioral Surveillance Survey (BSS) has been carried out three times -- in 2002, 2004 and 2006. The survey methodology uses a referral method, since sampling IDUs is no easy task. While providing good information, it does no present a representative sample of these people and when discussing the data, one must keep in mind various biases in the data, which may be present. However, the data points to trends that are generally positive, and it may be that Georgia will be able avoid a much larger HIV crisis.

So, what did the 2006 data show? Similar to 2004, the report found stability in the type of drugs injected. Most IDUs in the sample report injecting Subutex and also having started to use Antihistamines. In 2002, however, most users reported using heroin.

Also, in terms of good news, 96% of those who reported having paid sex (30% of the male sample -- about the same as in 2004), reported using a condom. The number who pay for sex and use a condom is substantially up from 2004, which hopefully indicates and increase in awareness of the dangers of unprotected sex. However, males users report not using condoms with their stable sex partner. In other good news, knowledge about HIV among drug users is also rising, at least in the sample, however remains low overall. Only 37% of the sample could correctly identify the six mechanisms of transmitting HIV. However, only 15% of the sample in 2002 could do so.

On the bad news front, respondents who inject seems to be getting younger and younger. There was a 10% jump from 2002 to 2006 in the those who started injecting between 15 and 19 years old.

Again, if you are interested in the whole report, please get in touch.

Carnegie Research Fellowship Program!

CRRC is happy to announce the Carnegie Research Fellowship Program. The program offers exceptional social science research opportunities in the United States for scholars from the Caucasus.

Specifically, scholars in the social sciences may apply for individual, non-degree research opportunities at universities and institutes in the United States. The program is directed at advanced researchers that already have a demonstrated track record in social science research. The research period lasts up to a full semester (i.e. 4 months), starting either September 2008 or January 2009.

Up to three scholars from the South Caucasus will be selected, to join a prestigious program administered by the National Council for Eurasian and East European Research (NCEEER -- that is an awful amount of "E"s). All costs for the scholars are covered, including round-trip airfare.

Applications need to be handed in as hard copies. Deadline for applications is April 30, 5 p.m., 2008, to be handed in to your local CRRC office. We suggest applicants study details in the guidelines and the application form closely, and in good time, to avoid disappointment. Please be aware that we will NOT be accepting applications in the humanities or international relations, since CRRC focuses primarily on empirical social research.

Note that the application process is very competitive, since a very concise research proposal is expected. We therefore encourage you to take advantage of specific mentoring that we will provide, to help you improve your application. The Carnegie Research Fellowship should present an extraordinary chance to researchers that can advance their work through a period of self-directed study in the US, and we will be happy to coach applicants.

Register your interest by writing to nana@crrccenters.org now, and no later than March 21, to learn how we can help you make your application competitive.

USAID Political Party Assessment of Europe and Eurasia

Admittedly we forgot to post this earlier, but we believe it is even more important with the upcoming elections in Georgia.

Democracy International, contracted by USAID, released a report on political party assistance across Europe and Eurasia. In order to carry out fieldwork, they selected four countries (Serbia, Romania, Georgia and Kyrgyzstan). There were a wide number of selection criteria variables, including the fact that both party institutes, the International Republican Institute (IRI) and the National Democratic Institute for International Affairs (NDI), had to be present in the country. In effect, this created an endogeneity problem in the research design, where countries were studied not because of their inherent differences but because the US government had invested the most resources in them. Of course, one country from each region was also chosen ensuring regional variation but also decreasing variation along other axes.
The report is long and includes an impressive theoretical overview. I will discuss only three subjects, which may be of interest to blog readers and discussed in the research:1) the relationship between political party assistance and development 2) survey work and 3) democratization in Georgia. However, I would encourage anyone interested in how USAID operates in the region to read the report and to also analyze it for the subtexts, since all documents put out by the government (even if it is contracted) will attempt to glaze over (or at least cover up) some things.

1
An important theoretical point was raised in the report: almost no literature exists on causality between political party assistance and development. While many other aspects of democracy have been thoroughly explored, most notably elections, how party assistance shapes democracy is a area ripe for study. I think one of the main reasons this has not been studied is the necessity of long-term on the ground fieldwork to trace out the processes of political party development. Since a comparative basis is needed to do so, the work involved seems particularly overwhelming. This could be an interesting place to create teams of local researchers to work on the ground in a well selected group of countries, to trace these processes over time in a standardized way.


2
The article highlights survey work, which is in one of the areas traditionally covered under party assistance and alludes to the fact that surveying has often been overemphasized and targeted not at domestic constituencies, but at the international and development community. Conversely, the report argues that the nexus between the survey work itself and any public policy outcomes is crucial and should be the focus of any polling efforts. If anything, the report underestimates the lack of sophistication that parties have in interpreting these results and the basic lack of understanding that they have in basic research methods. However, I would argue that the answer is not to give up on this work, but to work much harder on helping local government understand the basis of social research. I also think that does not emphasize enough the role between an active research community and public policy research. While the main goal of public opinion research is to inform politicians, the quality of the research often comes under fire in the region and there is no research community capable of demonstrating the quality of the polling and falsifying claims by polling companies that are unskilled. In order to add this concept to that of major international funders, would involve a much more holistic concept of the notion of development, which is generally missing from the debate. Also missing is any mention of the importance of mixing both quantitative and qualitative methods in polling, since the questions asked on the polls also may need to be better refined that they have been in the past.

3
In the Caucasus, Georgia was chosen for the analysis. Interestingly, the report, while of course lauding the Rose Revolution, notes many of the shortcomings of the the United National Movement (UNM) as a party. It uses the term "bandwagoning" usually used in the field of international relations to explain domestic Georgian politics, in which, according to the report, all of the major forces in the country including the media and civil society elites have become closely affiliated with UNM. According to the report, "these bandwagon effects in societies with weak democratic institutions can produce cycles of political convulsion where an initially liberally-oriented dominant party, facing few challenges from an effective opposition, loses dynamism and popular support,thereby engendering new rounds of political revolution." This statement is worrying for the future of Georgian democracy and may help to put the recent Okruashvili scandal into starker perspective.

Again, the full report can be found here.

Book Review | The Post-Soviet Wars: Rebellion, Ethnic Conflict and Nationhood in the Caucasus | Christoph Zürcher

The earliest books that came out about the Caucasus after the collapse of the Soviet Union were firsthand accounts of events. Now, a second spate of books, which attempt to apply analytical frameworks to the turbulent events that occurred have the breakup of the Soviet Union are beginning to appear. Christoph Zürcher’s The Post-Soviet Wars: Rebellion, Ethnic Conflict and Nationhood in the Caucasus, published with New York University Press, falls into this category. The book examines where wars occurred in the Caucasus (Georgia, Nagorno-Karabakh and Chechnya) and where they didn’t (Dagestan and Ajara) and places those cases studies within the context of the international quantitative literature that attempts to explain why internal wars occur.

Those who are knowledgeable about the Caucasus will find much information they have already come across. However, for those interested in international conflict who possess little regional understanding, the tersely written detail provides a good overview.

To whet your appetite for some of the details about why wars started in the Caucasus, Zürcher argues that, in Georgia, anti-Soviet rhetoric allowed for no maintenance of Soviet institutions, increasing the likelihood of conflict, since state institutions utterly collapsed as a result. Furthermore, the fallback on nationalist rhetoric, which was seen as the only way of creating a cohesive political force, then alienated both Abkhaz and Ossetians. Zürcher, perhaps controversially, also claims that Armenian politics looked very similar to Baltic politics (and different from Georgian and Chechen) in that the same type of state weakness did not exist. However, Zürcher makes the claim, which has been echoed in much of the democratization commentary about Armenia, that instead of the Baltic states’ orientation towards Europe, Armenia’s politicians unified around war in Nagorno-Karabakh, creating an anti-reform minded regime.

From a more technical standpoint, the book is a rare breed within the political science literature, as it is specifically concerned about testing existing theories about internal wars by examining a series of cases studies. In doing so the volume seeks to refine those theories. While this type of book is out of vogue because the academic nomenklatura does not perceive the endeavor as groundbreaking, it serves an important role in refining theories, something Zürcher does throughout the book.

So what does Zürcher find in relationship to the international quantitative literature? Several variables that are generally cited as determinants of internal war do not appear to hold true in the Caucasus: low economic development and mountainous terrain do not help in explaining the conflagrations in the Caucasus. Despite the Caucasus being mountainous, most conflict occurred in urban environs or in the plains. In the conflicts where mountains played a role, the guerillas (which conflict theory supposes are aided by mountains) had the mountains against them. In fact, Zürcher seeks to refine the theory about the relationship between mountains and war and suggests several plausible alternative hypotheses, part of the intellectual merit of the book. One interesting hypothesis is that mountains are a proxy for the cheap recruitment of male soldiers, since mountainous areas often have high unemployment rates and hence a male population ready to mobilize.

The volume also reinforces the idea found in the international quantitative literature that state weakness often plays an important role – perhaps much more so than underdevelopment – as does the role of one ethnic group constituting the majority of the population. This ethnicity argument is well-highlighted with Zürcher’s case study of Dagestan, where ethnicity did not play the same role as in Georgia, Armenia or Chechnya, in part because of the fact that no ethnic group had a majority.

Overall, this reviewer found the findings sound, but would have like to see more analysis of some of the interesting proxy variables discussed above. This, however, could form the basis of a new and fruitful conflict research agenda in the Caucasus.

This book review was also printed in The Georgian Times.

Inflation in Armenia? | Lecture by IMF Representative

Readers here may not be aware that actually our Armenian CRRC also runs its own blog, to announce and describe CRRC's events. One of the most recent events was a lecture by the IMF Resident Representative in Armenia, Dr. Nienke Oomes.

Dr. Oomes discussed what is happening to prices in Armenia, and offered a very comprehensive analysis. For a quick overview over the event, click here. Her PowerPoint presentation, which sets out her argument in good detail and includes four recommendations (more effective inflation targeting, facilitating reduction of import prices, tightening fiscal policy, and increasing the public's knowledge of inflation targeting), is available.

One snapshot:



Curious? The full presentation, with 38 slides, is right here.

Bertelsmann Transformation Index | Using a New Interactive Tool to Analyze the Caucasus

Many of our readers know of both our quibbles with indexes, but also our steadfastness when it comes to posting about them. The Bertelsmann Foundation released its trademark index, the Bertelsmann Transformation Index (BTI) (PDF), which according to its producers, is "the global ranking of the quality of democracy, the market economy and political leadership in 125 developing and transformation countries."

The BTI itself is a the example of what political scientists would call non-parsimonious. The BTI is a combination of two sub-indexes, which are in themselves made up of a wide number of indicators. These indicators in turn are based on an even greater number of variables. I don't want to bore you, so I won't go into details but refer you to the methodological report (which does a great job of documenting their approach). To boot, each of the 125 countries has a 25 page report.

However, despite its lack of parsimony, the BTI provides several good ways of visualizing the developments in the Caucasus -- though, as always, indices have their limits.

Scales run from 0 - 10, with 10 denoting the highest score. Georgia ranks 6.60 (38th) on the Status Index and 6.36 on the Management index (23rd). It gets strong upward ratings in the trends as well. Armenia follows behind Georgia with 6.14 (41stt) and 5.41 (56th). However, it shows no changes, in terms of trends. Azerbaijan lags behind with a score of 4.51 (87th) and 3.83 (99th) and also shows no significant change in terms of the trends.

The next three graphs show the seventeen main indicators that make up the ranking divided into the "Status" and "Management" parts broken down by country.






One way of simplifying and comparing the information in the BTI is a unique interactive tool, called the transformation atlas, which can be downloaded from the Bertelsmann website. While at first slightly difficult to interpret, the atlas provides a good visualization of individual countries, and tells a convincing story about many of the countries development problems. The averages, however, are less useful.



This image shows Georgia compared to Romania (Georgia in blue). The images shows neatly how uneven Georgia's development has been in comparison with a recent EU accession state like Romania. The places where the area in blue is small denotes a lower score. Georgia diverges significantly from Romania here on the key issues of rule of law, socioeconomic development, welfare regime, political and social integration and sustainability of transformation, among others. Since the sustainability variable pertains to the quality of education and research institutions in the country and environmental protection, we agree that Georgia needs more work on these areas. All of these areas sync well with what commentators have noted to be many of Georgia's shortcomings. Georgian does notably well on resource efficiency and steering capability, which address the ability to implement change -- something Georgians have certainly been doing.

The Armenia comparison shows the change over time function. What stands out in the Armenian case, as compared to the Georgian or even the Azerbaijani is the lack of change over time, particularly since 2003, where much greater change is noticeable in other countries in the region. Of interest here is that there has been no change in the socioeconomic ranking despite Armenia's status as the "Caucasian Tiger." However, the socioeconomic development indicator is more concerned with income gaps across the population and social exclusion, and all of the South Caucasus countries do poorly -- scoring 4 across the board.


A comparison of Azerbaijan and Kazakhstan yield interesting findings related to oil rich countries. The first obvious fact is that Azerbaijan looks theoretically very similar to Kazakhstan. However, Azerbaijan finds itself at a slightly lower level than Kazakhstan. Unfortunately, multiple comparisons cannot be carried out at the same time, so we cannot examine this longitudinally with multiple countries. Unsurprisingly, Azerbaijan scores high on stateness and economic performance and other economy related variables and extremely low on indicators related to democracy (upper left corner). Surprisingly Azerbaijan has higher than expected scores (4) on the rule of law.

We encourage our readers to explore more on their own!

Armenian Election | IREX Media Survey

IREX has released a print media election survey a few weeks ago, and this offers an interesting glimpse into growing political engagement in Armenia. Readers of our previous surveys may remember that our data has suggested widespread cynicism in Armenia. Well, as various commentators have noted, the entry of Levon Ter-Petrosian seems to have re-energized political debate.

The survey is an excellent overview, including media consumption habits, and it comes quite well documented. We're a little surprised how they managed to find so many male respondents (almost 50%, when typically women are overrepresented among any sample), and we also would have been curious to hear about the non-response rate, but these are minor issues.

The survey suggests that secrecy of the vote still remains a major issue. If 30% do not believe that their vote is secret, this already has an impact on the integrity of the election itself. Also, a similar number says that they will not do anything if they are disenfranchised, i.e. if they are not given the chance to vote at all. A solid 56% comes prepared, saying that they always bring their own pen to the elections (as otherwise, there may only be a pencil in the voting booth). Much more food for thought is available in the overview, here.

More broadly, though, recent developments in Armenia make one cautiously optimistic: there is political competition and there are substantive debates. Arguably, apathy is not the default setting, and an engaged population also constrains the space for extensive electoral manipulation.

The short overview of the survey is in the link above, and a longer version, with some SPSS charts should be available from IREX (or also us, upon request).

Data Initiative Snapshots | Reading

Books, of course, are essential to knowledge transmission, and for creating a broader conversation within a society. They may be hyped, but bestsellers such as Freakonomics or Wisdom of Crowds disseminate new ideas that enrich our understanding of the world. So it is interesting to follow up reading as an essential cultural practice. We asked the question in the Data Initiative 2007: did you read a book in the last six months? And here the results:
Armenia, clearly, reads most. It would be interesting to follow up reasons for this (and we will do a few crosstabs for future posts). One explanation for the prevalence of reading is that Russian is still very popular. If you just read in local language, the offerings are quickly exhausted and The Tipping Point is unlikely to be on your reading list anytime soon.

This is not just an arcane point: the societies still live in a sort of information vacuum, and breaking this will be essential to economic, social and political development.

Alpha Version of CRRC Data Initiative now online!!!

The alpha version of our Data Initiative data set, broad household data, covering lots of household data, but also political attitudes, social development, some health, education, migration, and social capital questions (and more) is online now. We interviewed more than 8000 people, so this really is the single largest dataset that is available on developments across the South Caucasus.

Register here to receive access to the dataset. In case you don't receive the confirmation email, let us know. (You typically should be able to log in with your email-address and password even without getting that confirmation email, though.)

You need SPSS to process it (trial versions of SPSS that last for 14 days can be downloaded here; a hefty 202 MB, though), and if you want to find out how to use SPSS, we offer a quick crude crash course on our website.

More updates on the dataset soon. And let us know what you find!!!

The Global Broadband Speed Test

According to CRRC's 2007 Data Initiative 2007 (visit www.crrccenters.org), around 3% of the population have Internet access at home in Georgia; nevertheless, we were curious to know how fast these people’s Internet speed is across the Caucasus. Speedtest.net, which is identical to Ookla Speed Test, provides visitors with the opportunity to measure broadband connections. For more, visit the site.

According to Speedtest.net, the top Internet service provider in Georgia is Rustavi 2 Online with 713 kb/s (it seems to be outdated though). The top region by download speed is strangely Dushetis Raioni with 640 kb/s (please let us know if you know why). In Armenia and Azerbaijan the top regions by download speed are Yerevan and Baku, respectively. The fastest download speed is in Azerbaijan — 3184 kb/s (only best test scores are used for the ranking, so this is probably inflated). Of course, the Caucasus still lags quite a ways behind developed countries. Japan is number one for download speed according to Speedtest.net — it scored 11,237 kb/s .

If you want to find out the fastest Internet Service Provider close to you or elsewhere, wish to install this software on your computer, or simply are curious, click here .

If you test it from the Caucasus or have criticisms of the test, we’d love to hear from you.

Georgian Borderlands | Mathijs Pelkmans

Many social researchers working on the Caucasus bemoan the lack of good scholarly works on the region. However, one recent book, which is both excellent and readable, seems to have fallen under people's radars -- Mathijs Pelkmans' Defending the Border: Identity, Religion, and Modernity in the Republic of Georgia, which came out in 2006 with Cornell University Press.

Pelkmans' book is deeply embedded within the literature on the studies of borderlands. Using the case of Sarpi (and Ajara more generally), Pelkmans argues convincingly that the Georgian (Soviet) border was not like other borders treated in the academic literature, which were porous and where strong cross-border networks have and continue to play an important role. Conversely, the Georgian border still plays a strong role, despite the ease with which it is now crossed.

Sarpi, which is only part of the study, provides a fascinating place to study a the effects of a Soviet border. First, the village was split in half after 1921. Second, the community is the only predominantly Laz community in Georgia. Therefore, in practice, the community should have felt more oriented towards their Laz brethren on the other side of the border in Turkey, where the majority of Laz live, after the border reopened.

However, the Soviet Union did something incredible with their tactics for closed border zones. Despite the fact that those on the Sarpi side of the village still have relatives on the other side of the border and their families also used to have landplots across the border, the Georgian Laz hardly ever go across into Turkey. Furthermore, only two marriages have occurred between the two Sarpis and those only in the heady days right after the border opening.

So what happened? Pelkmans' book examines three types of bordering, the literal border, the border between Islam and Christianity and the relationship between an urban provincial capital of Batumi and its rural periphery. As a brief insight into the Islam/Chrisitian divide, Pelkmans discusses the many people within the community of Sarpi who have now converted to Christianity as part of Tbilisi's narrative of the temporary conversion of its people to Islam under the Ottoman yoke, and the book contains wonderful quotes to highlight the process by which these people chose to convert to Christianity. Furthermore, Pelkmans examines the perceptions of the Turkish Sarpi "other." Those on the Georgian side of the village feel that their brethren on the Turkish side of the border have lost their Laz identity and become turkified. Indeed, they often refer to them as Turks. Conversely, as Pelkmans notes, the Georgian Laz have lost many of their cultural traits as well.

You will have to read the book, to get insight into the other types of bordering. However, in short, Pelkmans argues that religious, spatial and cultural borders have come together to create a border that still exists in the minds of the residents of Sarpi.

A follow up study on the other side of the village would prove fascinating, but for the time being Pelkman's account is a wonderful read.

Pre-Election Polls | what would be needed

With the election in Georgia approaching fast, polls are beginning to appear every week. Unfortunately, many of these polls are taken at face value. The reality is that at this point there is not a single pre-election poll that has demonstrated credibility. This does not necessarily mean that polling firms and newspapers are simply fabricating their data -- it simply means that if they were simply fabricating their data, it would be very difficult for anyone to know.

So can we be confident that a poll is credible? There are a number of basic stipulations:

1. Reveal the sampling methodology. How, in other words, do the pollsters ensure that interviewing a few thousand people is representative of the entire electorate? Choosing respondents requires a) knowing where most people live, and b) having a very strong theory about which people are likely to turn out to vote on election day. This is very difficult stuff, and even tiny errors here can have tremendous consequences.

2. Tell us about the field work. Were the interviews done face to face or by telephone? When and how? Did the survey enumerators explain who they were working for, and is it possible that the respondents knew that they were looking certain answers?

3. Publish the questionnaire. What exactly was asked, and how, and in what sequence?

4. Document the non-response rate. How many people refused to answer? There are plenty of people who don't pick up the phone, or who don't have 30 minutes to talk to pollsters...and in this country, many of those people will vote.

5. Allow peer-review. Power point presentations for nonspecialists are fine, but make the data set available to peers for professional scrutiny (and of course you can restrict usage). If you really are confident in what you're doing, this is the way to go.

If polls do not meet the standards, they really do not deserve to be taken seriously.

Too many commentators forget that the burden of proof is on the polling firms, not on the public. We seem to be entering a dangerous cycle, where there is a lot of awful information floating around, and no one has the ability to sort the good from the bad. This is as much a problem with what the public is demanding as what the firms are supplying. The public should beware, and commentators should be very cautious about taking firms' power point slides at face value, until some basic methodological questions are answered transparently.

Mapping Development | WRI's "Funnel the Money"

The World Resources Institute, a global environmental think tank based in Washington DC, is providing maps that allow a visual comparison of data for the countries in the South Caucasus. Called "Funnel the Money", it seeks to chart development within countries, and also track allocation of resources from the central government by providing regional comparisons. The intended users are decision-makers, development specialists and informed citizens, and this interactive tool tries to help them "allocate resources to reduce poverty".

The project remains in draft form, and, we hasten to add, is only as good as the source data. The interface is a little clunky, too.

Still, it is an entertaining way to explore what is going on. You were wondering about the number of Georgian centenarians (those over 100 years old), and how they are distributed across the country?

Here are the male centenarians according to the 2002 census:

And here are the female centenarians. Note that there are many more female centenarians, i.e. that the colors should not be compared to the illustration above.

So apparently there are as many female centenarians in Imereti as there are males in the entire country. And Adjaria has a high proportion of both male and female centenarians.

This is just one of many maps you can extract. What is particularly useful is that even the default shows three maps at the same time, allowing for a good visual comparison.

What would we like to be added? A slightly friendlier interface, an opportunity to compare countries, and maybe a disclaimer regarding data quality. As it is, this still seems to be designed from the data end, rather than from the perspective of the users.

The website notes that they are welcoming comments, a WRI team was just out to visit the Caucasus, so we are hoping for an updated version soon. For now, find the site here.

PISA Test | how are Azerbaijani schools doing?

OECD has just published their 2006 PISA results, which stands for "Program for International Student Assessment". In PISA, 15-year olds are tested for basic abilities in various fields. The 2006 round focused primarily on science learning. A little more than 60 countries participated, including Azerbaijan. Georgia and Armenia did not take part.

Alas, the news is sobering. While basic education still reaches the majority of the population, the quality of that education seems limited.

This is demonstrated, for example, by the proficiency levels on the science scale. About 20% of students in Azerbaijan only reach the first proficiency level. This is better than Argentina, Brazil and Tunisia (all 28% not managing to go beyond the first level), and way ahead of Quatar (48%) or Kyrgyzstan (58%). Arguably, Azerbaijan isn't even so far from Bulgaria (18%), Montenegro (17%), Romania (16%) or even Serbia (12%).

However, once a higher level is reached, Azerbaijani performance tails off. Only 0.4% of Azerbaijani students managed to reach Proficiency Level 3 [out of a total of 5 levels] -- that is a disappointing result, especially for a country that was part of a Soviet tradition of teaching. Even Kyrgyzstan is doing better (0.7%), as is Tunisia (1.0%), Quatar (1.6%), let alone Brazil (3.4%), Argentina (4.1%), Romania (4.2%), Chile (8%), Russian Federation (15%), USA (18%), with an OECD average of 20.3 and then that bunch of European states, including the Netherlands that have more than a quarter of their students reaching this proficiency level.

This is genuinely bad news: essentially science education in Azerbaijan has broken down, and lots needs to be done to even catch up. Note the strong divergence between Azerbaijan and Russia. And the same is true for reading ability: only 3.4% of Azerbaijani 15 year-olds reach the 3rd Proficiency Level (where the OECD average is about 28%).

While some of these results may be due to a lack of experience with testing, or even poor translation, the findings suggest where oil revenues could be invested to great use. On that level, it's commendable that Azerbaijan actually took part in PISA -- a very courageous step that yields concrete policy recommendations.

This is no more than a cursory analysis. The datasets are comprehensive and allow gender comparisons, as well as a review of various other indicators.

Exit Polls | a good idea?

With upcoming elections in Georgia, the attention is back on a theme that otherwise often gets neglected: what does the Georgian electorate want?

One of the ideas is to conduct an exit poll, to track the scale of any potential manipulation. You ask a representative sample coming out of the polling station who they voted for, and that should give you a good idea about the electoral results.

According to several people, the Georgian government very much would like such an exit poll. One reason, it is said, is that at least one opposition candidate is considering financing his own exit poll, and getting a large exit polls supported by international donors may counterbalance any biased results that have been paid for by a single candidate.

At face value, this seems like an attractive idea and it has a number of supporters. After all, all you're doing is triangulating, helping to verify what actually happened on E-Day and more information always seems better than less.

But where the trust in the election administration is limited, the risks of exit polls far outweigh any potential benefit. As the head of one organization working in the elections field pointed out, it would "be like fighting fire with fire".

Voters exiting a polling station may not actually tell the truth of who they voted for. This can have various reasons: the social acceptability of their choice, fear for jobs, the first impressdon that the interviewer makes, plain intimidation.

Therefore exit polls can easily be off by 5%, or more. Now imagine one candidate wins the first round legitimately with 52%, but the exit polls only show 47% support, because of such bias, or skewed sampling. The opposition will believe that the election has been stolen, although results simply were within the margin of error.

Ultimately there is no substitute for a regular, disciplined conduct of elections, with citizens actively participating to guard their own vote. If polls were an alternative, there would be no need for the entire elections rigmarole.

Labor Dynamics in Armenia | Youth Unemployment

In May 2007, the World Bank released a two volume report on Armenia's labor dynamics (click here for the overview page). Unfortunately, most of the report is based almost wholly on Armenian National Statistical Service (NSS) data from 2003 and 2004. Given the problems with Armenian statistical data and the fact that the statistics may already slightly outdated, the results should be read critically.

However, the volume has an exceptionally interesting chapter on "Youth Employment and Unemployment." The most immediate and striking fact is the idleness rate (defined as those who are neither working or in school) divided by the total youth population among youth aged 15-24 in Armenia. According to the NSS, over a third of young Armenians neither work nor study. Such a large cohort of unemployed youth is worrying since it prevents socialization into the labor market and may provide other negative social consequences, such as cycles of dependence and increased likelihood of drug and or alcohol addiction. Also, alarming is the passiveness of those who dropped out of the labor market; the majority of those who dropped out the labor market are not looking for a job.

Supporting findings from the CRRC Data Initiative, the World Banks reports data from the "Survey of Unemployed Youth" 2005. This survey found that networks play the most important role in finding a job. This, of course, puts a damper on incentive to complete professional training or improve skills and instead places a premium on increasing social connectedness. (So all that collective loitering outside the university may be worthwhile after all.)

The report, mirroring other reports, also found that particularly vocational and technical schools are not adapting to the needs of the job market, increasing the rate of dropout, since skill gained in these schools are viewed as useless. This situation is particularly grave, since these schools have no relationships with the job market.

Such high youth unemployment may be a large explanatory variable in the continuing outflow of migrants in Armenia. The question of labor migration is also addressed in the World Bank report.

Georgia's Performance? | Millenium Challenge Corporation's Meta-Index

With all the attention on Georgia, it may be interesting to revisit Georgia's most recent performance as seen by international organizations. As it happens, the Millennium Challenge Corporation offers a such an assessment through its annual scorecard, just released last week. This scorecard is a meta-index, drawing on data from the World Bank Institute, Freedom House, IFC, WHO, UNESCO and a few other organizations. There are three main categories: Ruling Justly, Investing in People, and Economic Freedom. Each of these break down into six subcategories, which seem well thought out. One indicator for "Investing in People", for example, is girls' primary education completion, which probably is a fairly good measure for more than just basic gender equality. This scorecard presents data from 2002 to 2006.

Generally, Georgia has been doing well. Most trends are going up, in particular in the Ruling Justly category. Control of corruption is increasing, the government is becoming more effective, and Freedom House suggests that political rights and civil liberties have been expanding (it will be interesting to see how they assess 2007: is that becoming darker, or indeed just flickering?). The World Bank Institute also suggests that the rule of law is improving -- maybe not something that everybody would agree with, but the chart concedes a margin of error that just about allows alternative interpretation.

There is a small dip in Voice and Accountability, where Georgia is above the median, but (according to data from the World Bank Institute) moving downward again in 2006.

Georgia does extremely well for land rights and access, being top of all the scored countries (IFC data). The same is true for business start up (IFC). Fiscal policy and inflation, however, have a downward trend, so Georgia is not scoring too well here.

In setting basic conditions for development, Georgia is and remains a success story. Obviously, the successes can be put at risk, since some of the achievements remain fragile. If we take the liberty to speculate, there are practical implications for current events. Politically, the opposition probably would be better served to move away from their hyperbole, and to concentrate on the problematic indicators (inflation, fiscal policy, voice and accountability), as well at the significant risks that the government is running, which may imperil all the other achievements so far. This would put them in line with the data on the ground.

One small puzzle is that supposedly girls' primary education completion is falling. Any idea why that may be the case? Is it just better data? Or migration?

The scorecards in general are exemplary in their presentation, with detailed datanotes, and a downloadable Excel overview (which reminds you that Georgia is a Lower Income Country, whereas Armenia and Azerbaijan are listed as Lower Middle Income Countries). Altogether an excellent shorthand introduction to governance in the respective countries.

Political Events in Georgia | Source of Dissatisfaction?

We normally leave political analysis to the many other qualified commentators. However, given current events, it is interesting to see that our Data Initiative shows that ever since 2004 there was a powerful trend of disenchantment in Georgia. Below, see the responses we received when asking "Do you think that things in our country are moving in the right direction?" Blue is positive, yellow negative. The data is for Tbilisi.


So what is wrong, in people's view? It is often reported that the protests are primarily based on economic disappointment following the revolution. There is some plausibility to that statement. In 2006, 40% of Georgian respondents told our interviewers that their economic situation basically had stayed the same. And 23% of respondents said that their situation actually had gotten worse.

However, Georgians overall do expect significant improvement: 34% were expecting that economic situation of their household will improve, while 17% believe things will get worse. That glass is half full: twice as many optimists is a significant proportion.

One friend suggested that the real reason for protest was not the social hardship. "Georgians", he said, "are used to living in difficult conditions, they can deal with that. What gets them really worked up is the sense of injustice and impunity." There is some corroboration for that view in CRRC's Tbilisi data.

Essentially this suggests that faith in the judiciary collapsed by 2006. Maybe this indeed is the more plausible explanation for the deep-rooted disenchantment. Perhaps people's expectations ultimately even were not so unrealistic: they expected to get a fair deal, not a great one. Who gets what when is less important than how this is decided. When the government failed to deliver on that rule-of-law expectation, and failed in some high-profile cases (notably the Girgviani killing), the patience began to wear thin.

There is a kernel of good news in this: the negative lesson of the recent events is that whatever you do in reform, the difficulty of reducing poverty will catch up with you. The analysis above suggests that an optimistic alternative interpretation is possible. As long as people believe that justice is being done, you probably can count on considerable patience. If the current Georgian government gets another lease of life, they may want to test that proposition.

2007 data will be released soon. Stay tuned.

Ethnology in Georgia | Kevin Tuite

The CESS Conference 2007 in Seattle in mid-October saw a range of papers and panels on the Caucasus. One of the most engaging presentations was delivered by Kevin Tuite, who teaches in the department of anthropology at the University of Montreal. Professor Tuite has been coming to Georgia since 1985, wrote his dissertation on "Number Agreement and Morphosyntactic Orientation in the Kartvelian Languages" and describes himself as an ethnolinguist.

His particular research interest are rites in the Georgian highlands, both in Svaneti and in Pshavi/Khevsureti. Unsurprisingly, celebrations are characterized by much drinking. But they also include less obvious moments, such as the turning of plates counter-clockwise before eating at a ceremonial feast.

While the highlands are set apart from Georgia, they are also markedly different. Svaneti has been largely christianised, whereas the Orthodox church has only had superficial impact in Northeastern Georgia, so that pagan rites still predominate. Now, Professor Tuite says, some of the traditions are beginning to wane through migration out of the harsh valleys, and some locals are turning back to ethnographic literature to rediscover their older practices.

What makes such a presentation stand out are the stories, the usual ethnographer's privilege. Perhaps the most entertaining account was that of a shrine in Pshavi/Khevsureti that is so sacred that even the priest (khevisperi) remains outside the fence and does not dare enter.

So how, then, did the priest get the blood of the sacrificed animal onto the shrine, as tradition demands? Trust local ingenuity: apparently the priest stood outside the fence, prepared three snowballs, slaughtered the animal over them, and then threw the three snowbloodballs over the fence at the shrine.

For a glimpse into this world, check Professor Tuite's website which provides an engaging account of his field trips, as well as access to his various publications. It is available at www.philologie.com.