What do CB interviewers’ ratings of respondents’ intelligence tell us?

CRRC’s Caucasus Barometer (CB) surveys regularly collect information about how the interviewers assess each of the conducted interviews – so called paradata that provides additional insight into the conditions surrounding the interviews (e.g., whether someone besides the respondent and the interviewer was present during the face-to-face interview), as well as interviewers’ subjective assessments of, for example, level of sincerity of the respondents. This type of knowledge is especially important not only because the populations of the South Caucasus countries often distrust polling and pollsters (Hayes et al., 2006), but also because the conditions during the face-to-face interviews are rarely perfect in the region. The latter usually stems from objective reasons, like, for example, crowded dwellings (especially in the winter, when families normally heat only a few rooms, where all household members tend to gather). This blog post looks at the information provided by CB 2013 Interviewer Assessment Forms in Armenia, Azerbaijan and Georgia, trying to determine whether there is a bias in interviewers’ assessments. Specifically, we will have a look at interviewers’ ratings of respondents’ intelligence.

Overall, CB interviewers tend to rate the majority of respondents as average to (moderately) intelligent, but rarely as “very intelligent”. Interestingly, Georgian interviewers’ ratings are more skewed towards the positive edge of the scale, compared to those in Armenia and Azerbaijan.

Note: The data was not weighted for the analysis performed for this blog post. 

These figures, of course, in no way provide objective assessments of the intelligence of the respondents – neither was it the aim of this exercise. We will now look at how these assessments correlate with respondents’ answers on their basic socio-economic characteristics, including gender, age and level of education.

In all countries, there is a strong positive correlation between interviewers’ assessment of intelligence of the respondents and the two variables measuring respondents’ level of education - highest level of education achieved by the respondent, and the years s/he spent in formal education. The strength of correlation is somewhat stronger in Azerbaijan (.549 for the first variable) and somewhat weaker in Georgia (.455) and Armenia (.468).

Azerbaijan is the only country where, according to CB 2013, there is a relatively weak, but significant correlation between interviewers’ assessment of respondents’ intelligence and the respondents’ gender (-.192), suggesting that there is a systematic tendency to rate male respondents’ intelligence higher than that of females’. This finding is all the more interesting since a bit over half of CB2013 interviewers in Azerbaijan were female (22 male interviewers, 24 female interviewers). In all countries, there is a tendency to rate the intelligence of those living in the capital cities higher than those living in other settlements – the respective correlation coefficients are not high (-.094 in Azerbaijan, -.119 in Georgia and -.155 in Armenia), but are significant. There is, however, no correlation between these assessments and respondents’ age.

Interviewers’ ratings of respondents’ intelligence do not seem to be straightforwardly related to the attitudes towards democracy, or reported trust towards major social and political institutions, although respective findings differ by countries. It’s only in Georgia that we can see weak, but negative correlation between trust towards the president of the country and assessment of respondent’s intelligence by interviewer (-.141), while the correlations between these variables are not significant in Armenia and Azerbaijan.  

What other variables might, in your opinion, affect an interviewer’s assessment of a respondent’s intelligence?

You can have a look at CB’s Interviewer Assessment Form at the end of each CB questionnaire, e.g., here; and, of course, you can learn all about CB data on CRRC’s Online Data Analysis tool.