[Note: This article was originally published here with OC-Media. It is available in Georgian here. This article was written by Meagan Neal, an International Fellow at CRRC-Georgia. The views expressed in this article do not necessarily reflect the views of CRRC-Georgia, the National Democratic Institute, or any affiliated entity.]
Around the world, an increase in the systematic collection of education data has revealed a previously invisible problem: even though more children have access to education than ever before, it does not necessarily mean they are learning. In fact, many students are progressing through school systems without mastering the basic skills necessary to fully participate in society. The gap is so stark it has become known as a ‘learning crisis’.
Given the global trends, a new wave of controversial reforms to Georgia’s school exam system, and a planned increase in the national education budget, what do Georgians think about the quality of their own education system? And should Georgia be concerned about its own students’ learning?
At first glance, public opinion suggests that education quality is not a large problem. Surveys CRRC and NDI conducted in December 2018 suggest that most people think the government is doing a good job providing education to the country’s children.
A plurality of Georgians (50%) thought that the government did a ‘somewhat good’ or ‘very good’ job providing education to all its citizens, while 34% rated the government’s performance as ‘somewhat bad’ or ‘very bad’. Among people with a child in their household currently in school, only 4% rated their family member’s school as ‘bad’ or ‘very bad’. Most said that it was ‘good’ or ‘very good’.
At the same time, most people also think that private tutoring is necessary to study well, to graduate from secondary school, or to get into university, suggesting Georgia’s schools are falling short of their basic duties.
As the chart below shows, a slight majority (55%) said that it was necessary to have a private tutor to pass secondary school graduation exams. A large majority (73%) said that tutoring was necessary to pass the Unified National Exams to gain entrance to university. Even respondents who rated their family member’s school as ‘good’ tended to agree.
These attitudes towards tutoring could change soon. In February, the government announced that they would abolish secondary school graduation exams and change the requirements for university entrance exams. According to the government, the new measures will allow secondary schools to focus less on test preparation, give universities more autonomy to choose which exams to use for their own admissions processes, and lower expenses for parents by reducing the need for private tutoring.
In March, the Prime Minister also announced that the government would increase education spending annually until 2022 — reaching a maximum of 6% of GDP, a quarter of the country’s budget.
The perceived necessity for tutors says something about the current quality of schooling, and these reforms may be missing the point. After all, abolishing tests will not fix the system if schools are not well-equipped to teach students the essentials of secondary school in the first place. Nor will pumping in more funding lead to better outcomes if those funds are not strategically allocated towards evidence-based strategies to improve learning, rather than an increase in ‘business as usual’ inputs like school infrastructure.
So how much are Georgian students actually learning? National assessment data can give some clues. Georgia has participated in the past few rounds of PISA, an international exam of 15-year-olds’ reading and maths abilities in OECD and partner countries.
The most recent results were not great. Half of the students scored below Level 2 in reading and maths, the PISA threshold for being able to fully participate in school and society. Even having made substantial improvements since 2009, Georgia was still one of the lowest performers among PISA-participating countries in 2015.
What’s more, trends from the 2009 round of PISA as well as the 2015 round of TIMSS (Trends in Mathematics and Science Study) suggest that most of the gains in performance are driven by urban students, while rural students continue to lag behind.
Can the new reforms help address these inequities? More funding and a reduced focus on national tests could be a good thing for low-performing students — if schools and teachers have the resources and capacity to redirect the energy and funds they would spend on test prep towards better adapting their curriculum and methods to the individual needs of their students, and if they are held accountable for the learning outcomes they deliver.
But right now, like in many countries, it is not clear Georgian schools have the capacity or accountability mechanisms to do so. As Transparency International recently highlighted, government spending on education has already continued to increase over the past eight years without any goals set around actual student learning. Abolishing exams, which, for all their imperfections, can be used as an accountability mechanism, is unlikely to help this.
Despite general perceptions that Georgia’s schools are fine, the data on student achievement and the perceived need for tutors suggest a much deeper problem of learning.
In theory, increased funding could be a great opportunity to address this. But unless the government pivots this spending towards strategies that have been shown to help students learn, such as more effective pedagogy and better teacher performance and accountability, it is unlikely to have much effect.
If they draw lessons from the global conversation around learning, on the other hand, the government can work to ensure that Georgia’s students develop the basic skills necessary to succeed in life — whether they need an exam to graduate or not.
Curious about perceptions of Georgia’s education system? The data used in the above article is available here.
Monday, March 18, 2019
Parents think Georgia’s schools are fine. But are students actually learning?
Thursday, February 26, 2015
Tracing regional inequalities in the Georgian education system (Part 2)
The first part of this blog post described the regional distribution of 2014 United National Exams (UNE) mean scores in Georgia. Here in the second part, we look at applicants’ gender and location (as explained below) in order to understand how mean 2014 UNE scores differ by these variables. This post also considers the role and quality of teachers in these regional disparities.
A location variable was generated for the present analysis, based on the applicants’ municipality of registration. This variable breaks down the applicants into five groups: those coming from the capital, large cities (Kutaisi, Rustavi, Batumi, Poti), municipalities in Western Georgia (including both urban and rural settlements), municipalities in Eastern Georgia (again, including both urban and rural settlements), and ‘other’ (IDPs and foreign-registered applicants). Although it is not possible to differentiate between urban and rural residents of the municipalities using UNE data, in general, municipalities included in the Eastern and Western Georgia groupings mainly consist of rural populations.
As descriptive analysis presented in the first part of this blog post showed, applicants from Tbilisi and large cities scored the highest. Applicants coming from ethnic minority municipalities and certain mountainous areas of Western Georgia received the lowest scores. In order to check whether regional differences are systematic or random, a statistical technique called Analysis of Variance (ANOVA) was employed. ANOVA checks whether the mean score of the groups under analysis differ from each other and whether a difference is statistically significant. The ANOVA results show that the main effects of gender, F(1, 26311)= 136.43, p<0.001, and location, F(4, 26311)= 396.36, p<0.001 are significant factors in applicants’ exam scores, while the interaction of these two variables is not significant, F(4, 26311)=1.67, p=0.1531. Post-hoc analysis demonstrates that there are no significant differences between the scores of applicants from the municipalities of Western and Eastern Georgia.
As shown in the chart above, in all locations, female applicants had higher mean scores than males. Applicants from Tbilisi – both males and females – were the most successful. Residents of large urban areas performed better than applicants from predominantly rural municipalities. Finally, applicants from the ‘other’ group scored higher than those from predominantly rural municipalities.
Although, in general, female applicants scored better than males, the difference was not vast. Importantly, there are significant regional gaps between applicants, which are clearly revealed by UNE scores. A number of reasons likely contribute to this disparity.
To start, Georgia is characterized by endemic regional inequality, including uneven quality of education in the country. There is an especially large gap between large urban areas and predominantly rural municipalities. The lack of quality education and good teachers in rural areas is obvious from the results of international tests taken by schoolchildren (TIMSS, PISA, PIRLS). Although, in these tests, Georgian pupils, overall, score around or above world averages, the picture is bleaker when looking at the scores by settlement type.
In Georgia, there is generally a lack of good teachers – nationally, over 90% of teachers of certain disciplines failed their certification exams in 2013. Still, the low quality of teachers in rural areas is more pronounced. If we look at the statistics for pupils per certified teachers (i.e. teachers who have passed special exams and hence are considered better performers compared with their peers), we see that this number is highest in ethnic minority municipalities – that is, there are fewer certified teachers with more students. In Marneuli, Ninotsminda and Akhalkalaki municipalities, there are one hundred or more pupils per certified teacher, while in Sachkhere municipality in Imereti the respective number is 25.
As school education, which should be instrumental in preparing pupils for university admission exams, appears to be inadequate, university applicants (and their parents) often hire private tutors rather than attend school in the final (11th and 12th) grades. Considering that private tutors are in many cases active teachers, the availability of quality tutors in rural areas is also lower, while better private tutors are found in the capital and large urban areas. This factor also contributes to regional disparities in UNE scores.
This series of blog posts explored the results of 2014 Unified Entrance Exams, taking into consideration regional and gender factors. Both descriptive and exploratory analysis shows that there are significant disparities between applicants, especially from the geographic point of view. While UNE was an excellent opportunity for many applicants who would not have had the chance to be accepted to a higher educational institution within the previous corrupt admissions system, certain segments of the population still do not enjoy equal opportunities, not because of the UNE per se, but due to the existing endemic problems that the Georgian secondary education system faces. In spite of its impressive success, the improved university admissions system has not tackled Georgia’s deep-rooted educational inequalities.
Have other insights? Join the conversation on our Facebook page or in the comments section below.
Monday, March 17, 2008
PISA in Azerbaijan | Take 2 | great maths scores
In a previous post
At the time of posting, we received some comments that the overall performance was not so bad. Azerbaijani math scores, it was pointed out, were much better. Time, therefore, for another look. Indeed, Azerbaijan performs much better at mathematics. (If you want to see what is being tested, check the PISA sample questions.)
Azerbaijan does better than, say, Argentina, Bulgaria, Mexico, Montenegro, and even Turkey. Conversely, the Baltic states and Russia do better than Azerbaijan. For example, Russia has about 15% reaching Level 4 in Mathematics, and about 6% reaching Level 5 (on a scale from 0 to 6, with 6 denoting highest). Azerbaijan by comparison only has about 7% reaching Level 4 in mathematics, and less than 1% getting to Level 5.
Still, altogether this is highly encouraging news. However, there is one item that is a little hard to explain, and if anyone has any ideas, let us know: according to this OECD data, Azerbaijan has the best basic mathematics training of all participating countries. Only 0.2% do NOT manage to reach the Level 1, which is quite exceptional. In Liechtenstein, for example, a wholesome 4% don't make it to Level 1, in Romania 24%, in Bulgaria nearly 30%, in Brazil even nearly 50%. So with 0.2%, what exactly happened in Azerbaijan? Is it really a case of no-child-left-behind? But what, then, should Switzerland (4.6%), Japan (4%) or Denmark (3.6%) learn from Azerbaijan?
Does anybody know? Did the bad students just not turn up for the test?
Friday, December 22, 2006
Schoolchildrens' Attitudes in Armenia: What Kind of Impact Has Civic Education Had?
Diana Ter-Stepanyan evaluated the effectiveness of the civic education training program implemented in Armenian high schools (upper grades of secondary schools). She conducted a quantitative (questionnaire based) survey among 494 tenth grade schoolchildren from all of
The results of the survey indicate that, despite having a course on civic education, Armenian schoolchildren still possess controversial opinions/attitudes on various legal issues. For instance, only 48% of surveyed schoolchildren were ready to unconditionally follow the laws, while the rest mentioned that they would only follow the laws if they reflected Armenian traditions and social values. Furthermore, only 40% of schoolchildren accepted the role of the court as a mechanism to regulate disputes.
Such research cries out for a governmental response.
In parallel with educating schoolchildren on civic education, Ter-Stepanyan recommended to convey certain values to the children which would motivate them to follow the laws. She also recommended providing training to the teachers of civic education and developing interactive training materials which would increase the effectiveness of the training sessions.
Information about Ter-Stepanyan’s work, along with other CRRC-Armenia fellows can be found here.
