PISA 2009 | Results for Azerbaijan

Every three years, a range of countries take part in the educational PISA tests, an assessment of the competencies of 15-year olds. The tests are organized by the OECD, and have led to soul-searching and vigorous educational reforms in various countries. In the 2009 round, 34 OECD countries and 41 partner countries took part.

Among the partner countries in 2009 is Azerbaijan. The news is dramatically bad. Azerbaijan ranked 74 out of 75 participating countries, coming in above Kyrgyzstan. Ahead of Azerbaijan are countries such as Jordan, Peru, Tunisia, Colombia, Thailand. It is not just a legacy of socialism: Russia is far ahead, just under Turkey and Lithuania, and not even far from Austria.The full table is here.

It would be interesting to find out more about the variation within Azerbaijan. There must be schools that are doing better. What can one learn from them? If this issue remains unaddressed, Azerbaijan's next generation will have little to show for all the oil wealth the country received.

Maths in Armenia | comparing through TIMSS

What is the average Armenian secondary school student’s competence in Maths and Science? Is Armenia doing fine, or is it time for the education policy makers to review the secondary school curricula. Trends in International Mathematics and Science Study (TIMSS) proposes an answer to these questions. TIMSS is an international evaluation of the mathematics and science knowledge of fourth and eighth grade students around the world. It was first conducted in 1995, and is administered once in every four years since then. In 2003, alongside with over 60 countries from around the world, Armenia also participated in the study.

Singapore (605), Hong Kong (586), Chinese Taipei (585), Japan (570) lead the TIMSS charts both in Maths and Science. The good news is that Armenia is among the countries that have high scores in Maths among the eight graders. Armenia is the 23rd in the chart, with an average score of 478 and is ahead of Serbia (477), Bulgaria (476), Romania (475) - countries that are above the international average score (467). The picture is different when it comes to the Maths scores of the fourth graders. Armenia’s indicators (456) are significantly lower than international average (495), leaving behind only Norway (451), Iran (389), Philippines (358), Morocco (341) and Tunisia (339).

Science scores among both the eight and fourth graders in Armenia are disappointing: 461 for the eight graders vs. 574 international average score and 437 vs. 489.

Interestingly, girls in Armenia show better results both in Maths and Science than boys (that's not the typical story). Look at the average grades below.

Subject	4th grade		8th grade
	Male	Female	Male	Female
Mathematics	450	462	473	483
Science	432	441	455	468

Some alternative studies conducted in Armenia suggest that TIMSS sample may not be representative of the overall population. If we understand the argument correctly, the authors of this study argue that students included in the sample in Armenia are from middle-upper classes. Effectively this could mean that the poorest remain underrepresented. This may be an interesting topic for research (any potential fellows out there?).

TIMSS methodology, datasets and the questionnaires are available for further analysis here. The study is not conducted in other countries yet, although Georgia looks poised to join TIMSS.

PISA in Azerbaijan | Take 2 | great maths scores

In a previous post we wrote about the PISA scores of 15-year olds in Azerbaijan. As you may recall, PISA is an international test of competency, primarily focusing on reading, mathematics and science. Azerbaijan deserves particular praise for participating in this challenging international exercise: the results in science were not altogether flattering, but it's better to take part than to stand aside, and it can only be hoped that Georgia and Armenia will also be taking part soon.

At the time of posting, we received some comments that the overall performance was not so bad. Azerbaijani math scores, it was pointed out, were much better. Time, therefore, for another look. Indeed, Azerbaijan performs much better at mathematics. (If you want to see what is being tested, check the PISA sample questions.)

Azerbaijan does better than, say, Argentina, Bulgaria, Mexico, Montenegro, and even Turkey. Conversely, the Baltic states and Russia do better than Azerbaijan. For example, Russia has about 15% reaching Level 4 in Mathematics, and about 6% reaching Level 5 (on a scale from 0 to 6, with 6 denoting highest). Azerbaijan by comparison only has about 7% reaching Level 4 in mathematics, and less than 1% getting to Level 5.

Still, altogether this is highly encouraging news. However, there is one item that is a little hard to explain, and if anyone has any ideas, let us know: according to this OECD data, Azerbaijan has the best basic mathematics training of all participating countries. Only 0.2% do NOT manage to reach the Level 1, which is quite exceptional. In Liechtenstein, for example, a wholesome 4% don't make it to Level 1, in Romania 24%, in Bulgaria nearly 30%, in Brazil even nearly 50%. So with 0.2%, what exactly happened in Azerbaijan? Is it really a case of no-child-left-behind? But what, then, should Switzerland (4.6%), Japan (4%) or Denmark (3.6%) learn from Azerbaijan?

Does anybody know? Did the bad students just not turn up for the test?

PISA Test | how are Azerbaijani schools doing?

OECD has just published their 2006 PISA results, which stands for "Program for International Student Assessment". In PISA, 15-year olds are tested for basic abilities in various fields. The 2006 round focused primarily on science learning. A little more than 60 countries participated, including Azerbaijan. Georgia and Armenia did not take part.

Alas, the news is sobering. While basic education still reaches the majority of the population, the quality of that education seems limited.

This is demonstrated, for example, by the proficiency levels on the science scale. About 20% of students in Azerbaijan only reach the first proficiency level. This is better than Argentina, Brazil and Tunisia (all 28% not managing to go beyond the first level), and way ahead of Quatar (48%) or Kyrgyzstan (58%). Arguably, Azerbaijan isn't even so far from Bulgaria (18%), Montenegro (17%), Romania (16%) or even Serbia (12%).

However, once a higher level is reached, Azerbaijani performance tails off. Only 0.4% of Azerbaijani students managed to reach Proficiency Level 3 [out of a total of 5 levels] -- that is a disappointing result, especially for a country that was part of a Soviet tradition of teaching. Even Kyrgyzstan is doing better (0.7%), as is Tunisia (1.0%), Quatar (1.6%), let alone Brazil (3.4%), Argentina (4.1%), Romania (4.2%), Chile (8%), Russian Federation (15%), USA (18%), with an OECD average of 20.3 and then that bunch of European states, including the Netherlands that have more than a quarter of their students reaching this proficiency level.

This is genuinely bad news: essentially science education in Azerbaijan has broken down, and lots needs to be done to even catch up. Note the strong divergence between Azerbaijan and Russia. And the same is true for reading ability: only 3.4% of Azerbaijani 15 year-olds reach the 3rd Proficiency Level (where the OECD average is about 28%).

While some of these results may be due to a lack of experience with testing, or even poor translation, the findings suggest where oil revenues could be invested to great use. On that level, it's commendable that Azerbaijan actually took part in PISA -- a very courageous step that yields concrete policy recommendations.

This is no more than a cursory analysis. The datasets are comprehensive and allow gender comparisons, as well as a review of various other indicators.