Parents think Georgia’s schools are fine. But are students actually learning?

[Note: This article was originally published here with OC-Media. It is available in Georgian here. This article was written by Meagan Neal, an International Fellow at CRRC-Georgia. The views expressed in this article do not necessarily reflect the views of CRRC-Georgia, the National Democratic Institute, or any affiliated entity.]

Around the world, an increase in the systematic collection of education data has revealed a previously invisible problem: even though more children have access to education than ever before, it does not necessarily mean they are learning. In fact, many students are progressing through school systems without mastering the basic skills necessary to fully participate in society. The gap is so stark it has become known as a ‘learning crisis’.

Given the global trends, a new wave of controversial reforms to Georgia’s school exam system, and a planned increase in the national education budget, what do Georgians think about the quality of their own education system? And should Georgia be concerned about its own students’ learning?

At first glance, public opinion suggests that education quality is not a large problem. Surveys CRRC and NDI conducted in December 2018 suggest that most people think the government is doing a good job providing education to the country’s children.

A plurality of Georgians (50%) thought that the government did a ‘somewhat good’ or ‘very good’ job providing education to all its citizens, while 34% rated the government’s performance as ‘somewhat bad’ or ‘very bad’. Among people with a child in their household currently in school, only 4% rated their family member’s school as ‘bad’ or ‘very bad’.  Most said that it was ‘good’ or ‘very good’.


At the same time, most people also think that private tutoring is necessary to study well, to graduate from secondary school, or to get into university, suggesting Georgia’s schools are falling short of their basic duties.
As the chart below shows, a slight majority (55%) said that it was necessary to have a private tutor to pass secondary school graduation exams. A large majority (73%) said that tutoring was necessary to pass the Unified National Exams to gain entrance to university. Even respondents who rated their family member’s school as ‘good’ tended to agree.


These attitudes towards tutoring could change soon. In February, the government announced that they would abolish secondary school graduation exams and change the requirements for university entrance exams. According to the government, the new measures will allow secondary schools to focus less on test preparation, give universities more autonomy to choose which exams to use for their own admissions processes, and lower expenses for parents by reducing the need for private tutoring.

In March, the Prime Minister also announced that the government would increase education spending annually until 2022 — reaching a maximum of 6% of GDP, a quarter of the country’s budget.

The perceived necessity for tutors says something about the current quality of schooling, and these reforms may be missing the point. After all, abolishing tests will not fix the system if schools are not well-equipped to teach students the essentials of secondary school in the first place. Nor will pumping in more funding lead to better outcomes if those funds are not strategically allocated towards evidence-based strategies to improve learning, rather than an increase in ‘business as usual’ inputs like school infrastructure.

So how much are Georgian students actually learning? National assessment data can give some clues. Georgia has participated in the past few rounds of PISA, an international exam of 15-year-olds’ reading and maths abilities in OECD and partner countries.

The most recent results were not great. Half of the students scored below Level 2 in reading and maths, the PISA threshold for being able to fully participate in school and society. Even having made substantial improvements since 2009, Georgia was still one of the lowest performers among PISA-participating countries in 2015.

What’s more, trends from the 2009 round of PISA as well as the 2015 round of TIMSS (Trends in Mathematics and Science Study) suggest that most of the gains in performance are driven by urban students, while rural students continue to lag behind.

Can the new reforms help address these inequities? More funding and a reduced focus on national tests could be a good thing for low-performing students — if schools and teachers have the resources and capacity to redirect the energy and funds they would spend on test prep towards better adapting their curriculum and methods to the individual needs of their students, and if they are held accountable for the learning outcomes they deliver.

But right now, like in many countries, it is not clear Georgian schools have the capacity or accountability mechanisms to do so. As Transparency International recently highlighted, government spending on education has already continued to increase over the past eight years without any goals set around actual student learning. Abolishing exams, which, for all their imperfections, can be used as an accountability mechanism, is unlikely to help this.

Despite general perceptions that Georgia’s schools are fine, the data on student achievement and the perceived need for tutors suggest a much deeper problem of learning.

In theory, increased funding could be a great opportunity to address this. But unless the government pivots this spending towards strategies that have been shown to help students learn, such as more effective pedagogy and better teacher performance and accountability, it is unlikely to have much effect.

If they draw lessons from the global conversation around learning, on the other hand, the government can work to ensure that Georgia’s students develop the basic skills necessary to succeed in life — whether they need an exam to graduate or not.

Curious about perceptions of Georgia’s education system? The data used in the above article is available here.

Which foreign language should children learn in schools in Georgia?

Since Georgia is a small country with a language that people outside the country rarely know, it is not surprising that people in Georgia want their children to know a foreign language. CRRC’s Caucasus Barometer (CB) survey has regularly asked about a foreign language which, in people’s opinion, should be mandatory in secondary schools in Georgia. Since 2009, a majority of people in Georgia have named English as such foreign language, followed, with a large gap, by the Russian language. Other languages were named by less than 2% of the population and less than 10% said that no foreign language should be mandatory.

In 2017, 69% of the population named English and 22% Russian as the foreign languages that should be mandatory in the secondary schools of Georgia. Even though English has consistently been named as the most wanted language, the share of the population naming this language decreased from 68% in 2011 to 52% in 2012, and then rebounded to 69% between 2012 and 2017. In 2012, the share of the population that named Russian as the most desired foreign language in secondary schools doubled compared to previous years and reached 32%. However, it decreased again by 10 percentage points between 2012 and 2017.



People of different ages and living in different settlement types have slightly different language preferences: 73% of people living in the capital or other urban settlements named English, while the share is lower (64%) in rural settlements. Moreover, young people tended to name English more often than older people.



The data also show that people who know English tend to suggest English should be a mandatory language in secondary schools more often than people who do not know English: 83% of those claiming to know English at an intermediate or advanced level said that this should be a mandatory language, compared to 63% of those who reported not to have basic knowledge of the language.



Despite the decline in 2012, the preference for English as a mandatory foreign language in Georgia’s schools is on the rise again. Those who know some English, live in the capital, and young people are more likely to support English being mandatory in Georgian schools.

To explore the data used in this blog post further, visit our Online Data Analysis platform.

Who in Georgia wants to study abroad?

Studying abroad can offer students the opportunity to learn new languages, travel, experience different cultures, and form relationships in addition to studying. The Knowledge of and Attitudes towards the European Union survey (EU Survey) implemented by CRRC-Georgia for Europe Foundation provides information about what share of the population in Georgia would like to go abroad to study, and the demographic characteristics of those who would like to.

Overall, almost a quarter of Georgia’s population (24%) reports a willingness to study abroad. Their median age is 29, and in this blog post we focus only on people who are between 18 and 58 years old, i.e. twice the median age. For the population in this age group, the share of those who report a willingness to study abroad increases to 33%. Most often, they name the US as the country where they would like to study.


Notably, slightly more females (37%) report being interested in going overseas to study than do males (30%). Of those who already hold a bachelor’s degree, 48% would like to go abroad to study, while of those who hold a master’s or higher degree, 39% want to study abroad.


Note: Options ‘Primary education’, ‘Incomplete secondary education’, ‘Completed secondary education’, ‘Secondary technical education’, and ‘Incomplete tertiary education’ were grouped into the category ‘Secondary education or lower’. Options ‘MA’ and ‘PhD student/PhD’ were grouped into the category ‘Master’s degree or higher.’ 

A willingness to go abroad to study is most common in the capital and least common in settlements with a large ethnic minority population. Notably, there is not much difference between the shares of people willing to study abroad in urban settlements outside Tbilisi and in rural settlements with a predominantly Georgian population.


Surprisingly, quite a large share of those who want to go abroad to study report no basic knowledge of English (37%). Thirty percent report they have intermediate and 15% - advanced knowledge of the language. This finding leads to some questions about whether those who report a willingness to study abroad would actually be able to do so. Notably, half of those who say they would like to study in the United States or the United Kingdom report either no basic knowledge or a beginner’s level of English.

This suggests the need for more focused efforts in the field of teaching foreign languages, and especially English.

To explore the data further, try CRRC’s online data analysis tool.

Education in Azerbaijan: A Snapshot

According to last month’s World Bank Azerbaijan Partnership Program Snapshot, there is an increased sense of urgency in Azerbaijan for long-term reform of the country’s education system. In its one-page education summary, the report describes Azerbaijan’s high primary school enrollment but details three important concerns: 1) low preschool enrollment, 2) low participation in higher education, and 3) an overall low quality of education.

According to Azerbaijan’s State Statistics Committee, only 16% of children between the ages of one and five attended preschool in 2012. However, there is a large gap in preschool attendence between urban and rural areas; in urban areas 23% of children aged 1-5 attend preschool, while this figure is 8.6% in rural areas. 

Created with data from the State Statistics Committee

The World Bank reports a slightly higher preschool enrollment rate in Azerbaijan at 27%, while the 2011 PIRLS-an international reading assessment study- reports that 36% of Azerbaijani fourth grade children attended preschool. Azerbaijan has one of the highest percentages of children without a preschool education among the 45 countries included in the PIRLS study.

*All data is from 2011 except the “Like countries” category which uses the latest available data from each country. (This was 2011 for all countries except 2012 for Kazakhstan, 2008 for Georgia, 2010 for Turkey, and Turkmenistan is not included because there is no data). Created with data from the World Bank. Downloaded from  http://data.worldbank.org/indicator/SE.PRE.ENRR/countries?display=default.

Low rates of preschool enrollment in Azerbaijan are particularly puzzling because Azerbaijan is also an upper middle income country according to the World Bank, yet it has a much lower preschool enrollment rate than many other upper middle income countries. Additionally, most Azerbaijanis place high value on preschool education. According to the the 2011 Caucasus Barometer, 85% of Azerbaijanis agreed that kindergarten is an important part of a child’s development. 

The Caucasus Research Resource Centers. (2011) “Caucasus Barometer". Retrieved through ODA - http://www.crrc.ge/oda/ on {May 2, 2013}. Available at http://www.crrc.ge/oda/?dataset=17&row=176

However, nearly 100% of Azerbaijani children attended primary school (grades 1 to 4) in 2011 according to Azerbaijan’s State Statistics Committee. This puts Azerbaijan on pace to achieve full primary school enrollment which is the major educational metric tracked by the Millennium Development Goals. World Bank data corroborates high primary school enrollment rates, but suggests that a small proportion of children (4%) may not attend school at all. The gap between gross and net enrollment suggests that some children attend primary school outside the suggested age range— they either enroll late or finish late.

 Primary School Enrollment in Azerbaijan

World Bank data on gross enrollment, net enrollment, and primary completion rate (male and female).

State Statistics Committee education data. *For a full explanation of gross and net enrollment see this helpful post by Friedrich Huebler.

According to the World Bank, Azerbaijan’s university enrollment rate is 20%— the second lowest in the CIS region and well below the averages for Europe and Central Asia, for upper middle income countries, and for “like countries” in Central Asia and the Caucasus. In fact, within the CIS only Uzbekistan (and possibly Turkmenistan but there is no data) shows a lower percentage of students attending university than Azerbaijan (9%). 

Available: http://data.worldbank.org/indicator/SE.TER.ENRR/countries?display=default.

Nevertheless, enrollment figures are increasing. The percent of secondary school graduates who have been accepted to university has risen from 21.7% to 27.4% since 2010. The number of students enrolling in university each year has risen 16% over the last three years, even though the percent of applicants has remained relatively constant at 60%. This means that more applicants are being accepted.

University Enrollment in Azerbaijan

State Students Admission Commission (SSAC) data compiled from annual reports available at http://www.tqdk.gov.az/az/statistics/.

Despite the fact that the ‘success’ rate for university applicants has risen by over 24% in the past three years, score distributions from the 2010-2012 State Students Admissions Commission university admissions exams show slightly decreasing scores during this time period. A possible explanation for the increasing number of students accepted with lower scores is that the test is becoming more difficult while the university admissions cut-off for lower scores has become more flexible. 

State Students Admission Commission (SSAC) data compiled from annual reports available at http://www.tqdk.gov.az/az/statistics/

Although Azerbaijan’s education system is making strides, there is still a lot of work to be done to create an inclusive system that provides opportunities for all children, and which develops Azerbaijan’s human capital. Azerbaijan’s spending on education has increased from 1.2 to 1.4 billion AZN between 2010 and 2012, but this is a decrease from 10% to 8.8% of the state’s total spending (because overall state spending has increased by a much higher percentage). Spending on education is currently 3.5% of the GDP, and while education spending is not considered to be a good indicator of education quality, the overall downward trend of social spending on education might be a problem. 


Contributed by Vitaly Radsky from the Center for Innovations in Education Policy Unit, and former CRRC International Fellow

Education in Georgia: Results of the 2011 Caucasus Barometer

Education is considered to be a crucial factor for social development. According to the 2011 Caucasus Barometer (CB) survey, much of the Georgian population considers education to be an important factor to get a good job in Georgia. Public interest in education is high and many Georgians think that the quality of secondary education has improved during the last five years.

The CB 2011 asked Georgians to choose the most interesting issue they would like to learn about from a provided list. Just over half of the population (55%) said they would like to get information about how much the government spent on education in 2010, which falls far above the interest in getting information about television stations’ owners (6%), political parties’ donors (5%), and government spending on military and defence in 2010 (6%). The high amount of interest in the government’s spending on education is one indicator of the high importance that the population ascribes to education.

Source: CB 2011

The 2009, 2010 and 2011 CB surveys asked Georgians what they consider to be the most important factor for getting a good job in Georgia. The data reveals an interesting trend. Much of the Georgian population attaches increasingly high importance to education, while the importance of social connections for getting a good job in Georgia has decreased during the past 3 years. In 2008 just under half (43%) of the adult population thought connections was the most important factor for getting a job, while this share decreased to 19% in 2011. In 2011, education became the most commonly mentioned factor for getting a good job in Georgia.

Source: CB 2009, 2010, 2011

Moreover, over half of the population (66%) thinks that the quality of secondary education has improved during the last five years and only 9% believe that it has worsened. A small part of Georgia’s population (9%) thinks that the quality of education has not changed at all and 15% either do not know or refused to answer the question.

Source: CB 2011

CRRC also asked Georgians what they think their level of English and Russian knowledge is and what foreign languages they think should be mandatory in secondary schools. Despite the fact that more people in Georgia report a better command of Russian, Georgians support having English as a mandatory subject in secondary schools (69%) - much more than Russian (14%). According to the 2011 CB, 62% of the population say they have no basic knowledge of English, while this share decreases to 9% in the case of Russian. 20% say they have intermediate or advanced knowledge of English, and 74% say the same for Russian.

Source: CB 2011

To sum up, education is gaining increasing importance and interest among the Georgian public who positively evaluate the quality of education in secondary schools relative to the past 5 years. CB data indicates that more Georgians consider education to be an important factor for getting a good job, and there is an increasing demand for English knowledge. In other words, the importance of education for getting a good job and knowledge of English is gradually replacing the importance of social connections for the former and the demand for knowledge of Russian with respect to the latter. 

You are also welcome to explore the newest data about education and other social issues via CRRC’s Online Data Analysis tool.

PISA 2009 | Results for Azerbaijan

Every three years, a range of countries take part in the educational PISA tests, an assessment of the competencies of 15-year olds. The tests are organized by the OECD, and have led to soul-searching and vigorous educational reforms in various countries. In the 2009 round, 34 OECD countries and 41 partner countries took part.

Among the partner countries in 2009 is Azerbaijan. The news is dramatically bad. Azerbaijan ranked 74 out of 75 participating countries, coming in above Kyrgyzstan. Ahead of Azerbaijan are countries such as Jordan, Peru, Tunisia, Colombia, Thailand. It is not just a legacy of socialism: Russia is far ahead, just under Turkey and Lithuania, and not even far from Austria.The full table is here.

It would be interesting to find out more about the variation within Azerbaijan. There must be schools that are doing better. What can one learn from them? If this issue remains unaddressed, Azerbaijan's next generation will have little to show for all the oil wealth the country received.

Introducing More Subtitling in Georgia | Policy Supported by Evidence

Georgian parliament is set to debate the introduction of broader subtitling (to replace dubbing) of foreign-language films on television. From the research point of view, this is excellent public policy. There may be some aesthetic disadvantages to subtitling, and viewers may take some time to get used to subtitles rather than voiceovers, but overall the educational advantages far outweigh any possible disadvantage.

Subtitling, rather than dubbing, has long been associated with superior foreign language ability. Countries that subtitle generally tend to be good in foreign languages. Examples include the Scandinavian countries, but also Netherlands. Several transition countries have taken the clever decision to go for subtitling, in order to further boost foreign-language ability. In Georgia (and the Caucasus generally), English is not spoken very widely.

http://mediadeskpoland.eu/upload/PP_Comite_media_280708_FINAL.pdf

Yellow: subtitles; Red: dubbing; Green: voiceover.

Countries that dub systematically tend to be relatively poor in foreign languages, examples including Germany (in spite of fairly high-quality teaching in schools), France, and Spain, and even worse in their pronounciation. Georgia has great opportunities if it develops extensive foreign language skills in English, maintains the skills in Russian, and ideally adds yet other languages (French, Persian, Turkish, German). Subtitling is one important step in that right direction.

The research evidence for this is strong. Danish and Romanian high school students, according to a 2003 Danish study, outperform German and Italian students in English comprehension. Denmark and Romania provide subtitles, Germany and Italy dub foreign-language films. In one study, 25% of Dutch primary school children said that they learned more English from television than at school. Scandinavian success in educational tests is consistently linked to subtitling on TV. Many other experimental setups have demonstrated the positive impact of subtitling on foreign language acquisition, including a better understanding of pronunciation (see Koolstra, 2002). Realistically, Georgia would have to invest dozens of millions to achieve a comparable impact through teaching across schools, and would still not reach the out-of-school population.

However, there is another important argument in favor of subtitling -- but one that is less obvious. Average families in Georgia watch television for around three hours per day. International studies have shown that television has a negative impact on children and youth -- but this impact could be reversed if children watch television in a foreign language, with subtitles. Studies demonstrate that television with subtitles helps children to learn reading (more strictly speaking: text recognition, or "decoding"), since they need to rely on subtitles to recognize what is being said (Koolstra, 1997). In other words, by subtitling, television becomes a powerful educational instrument that strengthens the link to written language per se, including one's own national language.

This draft law is an excellent public policy initiative by the government. It is cheap, comprehensive, turns a potential problem (TV consumption by children and youths) into a real asset, and could increase educational levels. From the research point of view, the available evidence suggests that subtitling deserves unqualified support.

Koolstra, C.M., Peeters, A.L., & Spinhof, H. (2002). The pros and cons of dubbing and subtitling. European Journal of Communication, 17(3), 325-354.

Koolstra, C.M., van der Voort, T.H.A., & van der Kamp, L.J.Th. (1997). Television's Impact on Children's Reading Comprehension and Decoding Skills: A 3-Year Panel Study. Reading Research Quarterly, 32(2), 128–152.

Math and Science in the South Caucasus | TIMSS 2007

TIMSS, Trends in International Mathematics and Science Study, has released their report for 2007. TIMSS is conducted every four years and it reports on mathematics and science education for 4th and 8th graders in 59 countries. In 2007 Georgia participated for the first time in the study. Armenia participated again in 2007 (you can read our previous blog posting about Armenia in TIMSS 2003 here).

So how did the countries of the South Caucuses fare? Armenia and Georgia fell below the international average in all subjects and grades except for one group. Armenian 4th graders performed just at the international average (500) in mathematics.



Armenia did much better than Georgia in both math and science, but both of the South Caucasus countries were outscored by several former Soviet and Eastern European countries. In 4th grade math scores for example; Kazakhstan (549) and Russian (544) ranked 4th and 5th overall. Several former Soviet and Eastern European countries performed at or above the international average including: Latvia (537), Lithuania (530), Hungary (510), Slovenia (502) and Armenia (500). Below the international average were the Slovak Republic (496), the Czech Republic (486), Ukraine (469) and Georgia (438).

Georgia was outperformed by all other participating former Soviet Countries and Eastern European countries in both math and science for 4th and 8th graders. What should be done? Reader input and discussion is most welcome. How much the ongoing reforms have showed up in the test results in the Georgian case would be worth of further study.

On a positive note, Armenia is doing better than it was four years ago in all categories and at all grade levels.

Armenia


The full 2007 TIMSS report is available here.

Restructuring Schools in Armenian Neighborhoods: Does Social Capital Matter?

Public schools in Yerevan face serious problems of restructuring. Most of the schools have not been renovated since the collapse of the Soviet Union. Does economic well-being affect the level of social capital in the neighborhood? Are the neighborhoods with higher social capital more likely to be willing to participate in school renovations? The answer depends not only on public cooperation, but also on socio-economic well-being. Armine Asryan together with her partner Anush Davtian investigated social capital in four neighborhoods of Yerevan.

According to the researchers, income positively determines the level of social capital -- the higher the income, the higher the social capital; and there is a further relationship between social capital and school renovation -- the higher the social capital, the more likely it is that the community will take part in school renovation. Armine characterized the communities as having low bridging and high bonding capital, which indicate low civic participation apathy and extreme individualism among those four communities.

The researchers developed policy recommendations such as enhancing the transparency of school boards. The data show that most of the respondents who expressed their willingness to support school restructuring affirm that they donate money through school boards. Therefore, clear and continuous reports on the management of the funds will enhance parents' participation in school renovation projects.

The paper is posted on the CRRC-Armenia website. Please let us know what you think.

No Adult Male Role Models: Distorting Armenian Male Teenager’s View of Masculinity

Gender issues in Armenia are currently under-evaluated and are interpreted predominantly as women's issues. Most of the recommendations drawn from different research suggest special policies to support and reinforce women's integration into traditionally male-dominated areas. According to CRRC-Armenia fellow Mariam Martirosyan, it is dangerous to ignore areas where significant under-representation of men is apparent; since in the long-term view, it may lead to a outcome for the Armenian male, the Armenian family unit and Armenian society in general.

Mariam Martirosyan studied the impact of the lack of adult male role models or senior male mentorship in Armenian schools on male teenagers' perceptions of masculinity. The research findings were alarming. The lack of male teachers in contemporary Armenian schools causes misperceptions on masculinity and male gender values among teenagers, often resulting in increased crime rates and delinquent behavior among young male adolescents.

In the attempt to find solutions to the problem presented in the paper, the fellow recommended to attract and engage more men into Armenia's secondary schools by increasing teachers' salaries; to bridge adult and young males via programs like the ZANG program, to assign male students of higher education institutions as mentors for primary, secondary and high-school students; to organize frequent tours to the army or to different factories dealing with technology and construction.

The fellow published an article in the June 2008 issue on "Journal of Education and Human Development" at the Scientific Journals International. The paper (in PDF format) is also available for download at the CRRC-Armenia website.

Caucasus Data | Language: Russian versus English?

Recently, we happened upon an article that talks about the use of Russian across the Caucasus. Is Russian becoming obsolete? According to the article, some Georgian politicians suggest this is the case. At the same time, the article points out that the uptake of English is too slow to replace Russian as a lingua franca.

As usual, we were a little disappointed that the article relied on many assertions, without actually checking for data. As it happens, the Data Initiative provides some insights on the question.

With English-language use, this is what respondents told us about their language ability:


One of the explanations for the different levels may be that diaspora contacts have traditionally brought more English to Armenia. Conversely, economic growth in Baku has attracted people more recently to begin learning English (as you will notice in many hotels and restaurants).

But then, where do people see language going in the next few years? Interestingly, there's overwhelming approval for English as the first mandatory foreign language in secondary schools. Almost 70% in Azerbaijan and Georgia would prefer English, with 57% favoring English in Armenia. In Armenia, 37% suggest Russian should be mandatory in secondary schools. Only 27% of Georgians, and, surprisingly, only 18% Azerbaijani respondents prefer Russian. In Azerbaijan, 9% think that no foreign language should be mandatory, while 4% suggest that Turkish should be the first foreign language.

It is at this point, that we're curious to explore how the respondents break down into different groups. When looking at the data, Nana stumbled upon an interesting dimension: the more intelligent the respondent is rated to be (by the interviewer, and it's a very crude measure, but returns interesting results), the more they favor English.


Want to know more? Compare women and men? The old and the young? Citydwellers' English versus rural folk? Let us know, or, better, explore the data set yourself.

Maths in Armenia | comparing through TIMSS

What is the average Armenian secondary school student’s competence in Maths and Science? Is Armenia doing fine, or is it time for the education policy makers to review the secondary school curricula. Trends in International Mathematics and Science Study (TIMSS) proposes an answer to these questions. TIMSS is an international evaluation of the mathematics and science knowledge of fourth and eighth grade students around the world. It was first conducted in 1995, and is administered once in every four years since then. In 2003, alongside with over 60 countries from around the world, Armenia also participated in the study.

Singapore (605), Hong Kong (586), Chinese Taipei (585), Japan (570) lead the TIMSS charts both in Maths and Science. The good news is that Armenia is among the countries that have high scores in Maths among the eight graders. Armenia is the 23rd in the chart, with an average score of 478 and is ahead of Serbia (477), Bulgaria (476), Romania (475) - countries that are above the international average score (467). The picture is different when it comes to the Maths scores of the fourth graders. Armenia’s indicators (456) are significantly lower than international average (495), leaving behind only Norway (451), Iran (389), Philippines (358), Morocco (341) and Tunisia (339).

Science scores among both the eight and fourth graders in Armenia are disappointing: 461 for the eight graders vs. 574 international average score and 437 vs. 489.

Interestingly, girls in Armenia show better results both in Maths and Science than boys (that's not the typical story). Look at the average grades below.

Subject	4th grade		8th grade
	Male	Female	Male	Female
Mathematics	450	462	473	483
Science	432	441	455	468

Some alternative studies conducted in Armenia suggest that TIMSS sample may not be representative of the overall population. If we understand the argument correctly, the authors of this study argue that students included in the sample in Armenia are from middle-upper classes. Effectively this could mean that the poorest remain underrepresented. This may be an interesting topic for research (any potential fellows out there?).

TIMSS methodology, datasets and the questionnaires are available for further analysis here. The study is not conducted in other countries yet, although Georgia looks poised to join TIMSS.

PISA in Azerbaijan | Take 2 | great maths scores

In a previous post we wrote about the PISA scores of 15-year olds in Azerbaijan. As you may recall, PISA is an international test of competency, primarily focusing on reading, mathematics and science. Azerbaijan deserves particular praise for participating in this challenging international exercise: the results in science were not altogether flattering, but it's better to take part than to stand aside, and it can only be hoped that Georgia and Armenia will also be taking part soon.

At the time of posting, we received some comments that the overall performance was not so bad. Azerbaijani math scores, it was pointed out, were much better. Time, therefore, for another look. Indeed, Azerbaijan performs much better at mathematics. (If you want to see what is being tested, check the PISA sample questions.)

Azerbaijan does better than, say, Argentina, Bulgaria, Mexico, Montenegro, and even Turkey. Conversely, the Baltic states and Russia do better than Azerbaijan. For example, Russia has about 15% reaching Level 4 in Mathematics, and about 6% reaching Level 5 (on a scale from 0 to 6, with 6 denoting highest). Azerbaijan by comparison only has about 7% reaching Level 4 in mathematics, and less than 1% getting to Level 5.

Still, altogether this is highly encouraging news. However, there is one item that is a little hard to explain, and if anyone has any ideas, let us know: according to this OECD data, Azerbaijan has the best basic mathematics training of all participating countries. Only 0.2% do NOT manage to reach the Level 1, which is quite exceptional. In Liechtenstein, for example, a wholesome 4% don't make it to Level 1, in Romania 24%, in Bulgaria nearly 30%, in Brazil even nearly 50%. So with 0.2%, what exactly happened in Azerbaijan? Is it really a case of no-child-left-behind? But what, then, should Switzerland (4.6%), Japan (4%) or Denmark (3.6%) learn from Azerbaijan?

Does anybody know? Did the bad students just not turn up for the test?