Safer transport options for passengers: Recommendations

In the previous blog posts in this series (see here, here, here, and here), we reported the design and results of a randomized control trial on minibus safety in Georgia. In this blog post, we provide recommendations for the Government of Georgia based on the results of the experiment. First, we recommend that the Government:

• Create an anonymous minibus monitoring program

Telling minibus drivers that they are being monitored for safe driving and may be punished for safety-violations may lead to safer, less distracted driving. The results of the experiment suggest such a program would likely be effective. This suggests that the government has an opportunity to implement a small program, which could have an important impact on making minibus driving safer and reduce the number of accidents related to dangerous driving. If the government does in fact implement such a program, we recommend that the program:

• Fine unsafe minibus drivers

While our experiment could not test the impact of a potential fine for unsafe driving, the behavioral economics literature suggests that individuals are roughly twice as likely to avoid losses as they are to seek out gains. Given that losses have stronger effects, this is also likely to ensure that there actually is an overall effect of the program. Importantly, this may offset costs associated with the program.

• Publicize the program in the lead up to implementation

Minibus drivers should be made aware of the program. If they do not know that they could be monitored, the program will be slower to encourage safe driving.

• Select routes for monitoring randomly on a daily basis

This would help prevent drivers from driving artificially safely in order to avoid a fine on a trip when they believe a monitor to be present based on prior information.

• Use few monitors, and change them regularly

The success of such a monitoring program relies on monitors being able to maintain their anonymity. In a country like Georgia, with a small population and dense social networks, maintaining monitor anonymity will be challenging. Hence, the government should consider drawing monitors from one of the civil service agencies with a relatively large staff. The Ministry of Internal Affairs Patrol Police Department would likely be an ideal institution given that patrol police officers are already aware of road safety legislation, and there are a sufficiently large number of officers who could participate in the program on a rolling basis.

The above recommendations are likely to help improve the safety of marshutka driving in Georgia. With less distracted and dangerous driving on Georgia’s roads, the high level of accidents, injuries, and fatalities on roadways are likely to decline. Ultimately, such a policy is likely to lead to safer transport options for passengers.

Safer transport options for passengers: Immediate, lasting, and contagion effects

As the last two blog posts in this series have described, CRRC-Georgia carried out a randomized control trial on minibus safety. This post reports the results of the experiment, and specifically whether there were immediate effects of being monitored, lasting effects on drivers which were monitored, and whether drivers who were not aware that they were being monitored ended up driving safer as a result of contagion effects i.e. drivers talking to each other about the monitoring. Overall, the results suggest that a small minibus monitoring program is likely to decrease dangerous driving behaviors in Georgia.

Did active knowledge that an individual was being monitored matter? To find out, we compare the control group from the first wave with the treatment group observations from the second wave. The statistical analysis suggests that drivers made 1.4 fewer calls per trip, smoked 1.5 fewer cigarettes, made 2.2 fewer illegal passes, and 2 fewer aggressive maneuvers on average. Overall, in the group that was directly aware that it was being monitored, there were 7.2 fewer incidents on average per trip. Other measures such as speed and seat belt non-use did not decline or increase in a statistically significant manner.

Based on these statistics we can conclude that direct knowledge of being monitored lead to fewer distracted and other dangerous driving behaviors. However, did these effects last? To understand whether direct knowledge of being monitored and that one would again be monitored had a lasting effect, we compare the results of the drivers who knew they were being monitored to roughly two weeks later when they did not know they were being monitored. In this case, a lasting effect is present if there was a statistically significant decline in a behavior in the first round of monitoring, and no significant change in the second round of monitoring.

Data analysis suggests that drivers who knew they would be monitored maintained lower levels of smoking, telephone conversations, illegal passing, aggressive maneuvers, and number of incidents overall. However, it is important to note that the drivers did drive less safely than they did when they were aware of being monitored. The differences between waves are however not statistically significant, suggesting that drivers did in fact drive safer roughly two weeks after being aware that they were being observed and would be again.

When it comes to contagion, the results of the statistical tests uniformly show no significant change except for in speed. Given that this is one in ten tests and that there was no significant effect from the first wave of monitoring on speed, we suspect that this test may have been found to be significant based on chance or other extraneous factors.

The table below presents the average treatment effect on the treated for each indicator with Abadie Imbens standard errors in parenthesis. On the line following the indicator name, 95% confidence intervals are provided. One star indicates that the estimate has less than a one in twenty chance of being due to chance alone. Two stars indicates that the estimate has less than a one in one hundred chance of being due to chance alone, and three stars indicates that the estimate has less than a one in one thousand chance of being due to chance alone. No stars indicates no significant difference.

Based on the above statistics we can conclude that direct knowledge of being monitored led to fewer distracted and other dangerous driving behaviors. However, the combination of the lack of significant contagion effect in combination with the increased but not significant increase between treatment groups suggests that a comparison of descriptive statistics is important to gain a sense of whether there was a lasting effect. The table below shows the average of each indicator given above in each group of the experiment.

The average indicator for the third wave of treatment and control group observations are quite similar. The similarity between means suggests one of two things. First, there may have been a small contagion effect since the levels are lower than in the first round control as well as a lasting effect that was not statistically significant. Supporting this conclusion is that both second wave means are lower than the mean in the first observation of the control group. The second possibility, however, is a lack of lasting effect and that an external factor caused the lowering of indicators during the third wave of observation.

Overall, the results of the experiment suggest that direct knowledge of being observed lowers the number of dangerous and distracted driving behaviors by a significant amount. This experience may have a lasting effect on drivers. What does this mean for the policy we proposed in the previous post, and what can the government do based on this information? In the next post, we provide recommendations for the government based on the information presented in this post.

Safer transport options for passengers: A Nudge on Marshutka Safety

Within the auspices of the Safer Transit Options for Passenger’s project, CRRC-Georgia carried out a randomized control trial, attempting to test whether a small policy change could make a large difference to minibus driving safety in Georgia. What was that small change, and how did we measure whether it might matter? This blog post provides an overview of both the policy to be tested and how we measured whether it could work.

The policy

A potentially simple way of decreasing distracted and other dangerous driving practices among minibus drivers is to use anonymous monitoring combined with penalties for dangerous driving. Under such a policy, the government would hire a small number of monitors to ride on randomly selected minibuses throughout the country without informing the drivers. After the ride, the monitor would report on any serious road safety violation as well as the number of distracted driving activities and safety violations carried out. If the driver committed serious traffic violations fines could be given out.

This policy would encourage safe driving and discourage dangerous driving. Moreover, it would require a relatively limited amount of funds from the government. Assuming that only 10 monitors are engaged in the program and they work 200 days a year, they could easily make up to 4000 trips, covering the majority, if not every, minibus route in the country. By randomly assigning monitors to different routes, minibus drivers would not be able to predict whether they are or would be monitored on any given trip. Hence, with the credible risk of being fined, drivers would likely drive safer.

Testing the theory

While the above policy, in theory, is quite sound, practice and theory often diverge. Hence, in order to test whether the policy would in fact be effective, CRRC-Georgia carried out a randomized control trial, which tested whether the knowledge that drivers might be monitored for safe driving and drivers could receive an award might improve their driving in late 2016.

Randomized control trials are a research design that comes from medical research. When doctors are attempting to understand whether a new medicine is effective, they randomly assign whether a patient gets the new drug or a placebo. Randomization is used to try to eliminate whether confounding factors that may make the medicine (in)effective for different individuals are distributed equally between the groups that receive the medicine and the placebo. Following this logic, for the STOP experiment, we randomly assigned minibus drivers to either a treatment or control group.

While in medicine, a treatment is, well a medical treatment, in our case information and action functioned as treatments. Minibus drivers that were assigned to the treatment group were told that they were being observed on a number of driving safety measures, and that the safest drivers would be awarded a gas voucher.  Importantly, as an NGO, we could not make a credible claim about the issuance of fines. Hence, we were unable to fully test the effectiveness of the proposed policy above. Notably, we would expect the issuance of fines for unsafe driving to be a stronger incentive for individuals to drive safer, because people are usually more averse to losses than prone to seeking gains, a phenomenon psychologists refer to as loss aversion.

Between September 20th and October 20th, CRRC-Georgia interviewers observed 360 minibus trips in three waves of observation. In the first wave of observation, minibus routes which had been randomly selected were observed without telling the driver that they were being monitored. This group forms the study’s control group. In the second wave of observation, routes assigned to the treatment group were observed. In the third wave of observation, observers returned to both the control and treatment minibuses for anonymous observation.

Minibus drivers in the treatment group who drive along similar routes were informed that:

1. Their trip would be monitored for safety along a number of dimensions;
2. A monitor would return in the coming weeks and monitor their driving as well as other drivers again without telling them;
3. If they were found to be among the safest drivers, they would be rewarded with a petrol voucher.

Over the course of the trip, monitors in all three waves recorded how many times drivers:

1. Smoked;
2. Text messaged;
3. Had telephone conversations;
4. Did not wear a seat belt;
5. Passed in areas it was not legal to do so;
6. Made other aggressive driving maneuvers;
7. Behaved aggressively towards passengers;
8. Behaved aggressively towards non-passengers.

Monitors also recorded stop and travel time, whether additional seats were added to the bus, whether passengers stood during the ride as well as a number of characteristics about the state of the minibus. Following the trip, the average speed of travel was calculated.

The above research design allows for a number of comparisons. First, by comparing the first wave control group to the second wave treatment group, we can test how the direct knowledge that one is being observed for safe driving effects driver safety. Second, by comparing the second wave treatment group to the third wave treatment group, we are able to tell whether the knowledge that one might be monitored again in the near future would have some lasting effect. Third, by comparing drivers in the first wave control group to drivers in the third wave control group, we can tell whether there was a contagion effect from the experiment i.e., whether the treated drivers talked to the non-treated drivers about the monitoring and they in turn also became safer drivers.

To test for the above types of effects, we used multivariate matching with genetic weights and calculations of average treatment on the treated (ATT). For readers interested in these methods, please see here. Using these methods, we calculate both the size of an effect and the probability that it emerged by chance alone.

In the next post in this series, which will appear tomorrow, we report the results of the experiment.

Safer transport options for passengers: Distracted and dangerous driving among minibuses

Over the course of this week, CRRC-Georgia will publish the results of a randomized control trial on minibus safety. While the introduction post to this series highlighted that Georgia’s roadways are dangerous, just how dangerous minibus drivers are has largely been left undescribed. As part of the Safer Transport Options for Passengers project, CRRC-Georgia collected data on dangerous and distracted driving practices on minibuses. This blog post reports descriptive statistics about distracted and dangerous driving from drivers unaware that they were being monitored.

While estimates do not exist for Georgia for how many accidents are caused by distracted and other dangerous driving practices, they are very likely contribute to the high fatality rates on the roads in Georgia. When it comes to distracted driving alone, studies from other contexts suggest that a distracted driver’s chances of being in an accident are four times higher. Cell phone use is associated with increased incidence of accidents among both novice and experienced drivers. Importantly, commercial vehicle drivers are no exception, with increased risk of accident associated with distracted driving among commercial drivers.

Overall, among minibus drivers unaware of being monitored, 96% engaged in some form of poor driving behavior, with illegal passes being the most common, followed by making telephone calls, and other aggressive driving maneuvers. Notably, few drivers were observed text messaging while driving. While we cannot claim that the data presented here is representative, for technical reasons, it is highly suggestive of the high prevalence of dangerous and distracted driving practices among minibuses on Georgia’s roadways.

The graph below presents the average number of times a driver engaged in a dangerous driving behavior as well as the maximum value for each indicator. Although the minimum value was 0 for each indicator, the high values for the maximums suggest that some drivers are particularly prone to dangerous driving, with as many as 62 dangerous events observed in a single trip.

The above statistics suggest that distracted and dangerous driving are common problems among minibus drivers in Georgia. In order to help ameliorate the situation, CRRC-Georgia tested whether a simple minibus monitoring policy would decrease the prevalence of dangerous and distracted driving. In subsequent posts in this series, we report the results of the randomized control trial, which suggest that such a program would in fact be effective.

Safer transport options for passengers: Series introduction

Complaints about minibus (marshutka) driving are common in Georgia. From excessive speeds, to erratic and distracted driving, minibus travel in Georgia is often less than safe and comfortable. But could a small change make an important difference in passenger safety and comfort? At CRRC-Georgia, we wanted to find out. Hence, CRRC-Georgia together with the Fund Partnership for Road Safety carried out the Safer Transit Options for Passengers (STOP) project within the East-West Management Institute’s (EWMI) Advancing CSO Capacities and Engaging Society for Sustainability (ACCESS) project, funded by the United States Agency for International Development (USAID). Within the project, we carried out a randomized control trial, which tested whether a small incentive – a gas voucher for safe driving – and telling minibus drivers that they would be monitored could improve marshutka driving. In the coming weeks, we will report the results of the randomized control trial and institutional analysis carried out within the project. In today’s post, we provide some background on road safety in Georgia.

According to the World Health Organization (WHO), Georgia has among the highest death rates on roadways in the wider European space.* The problem also appears to be getting worse: official figures from the Ministry of Internal Affairs of Georgia show an increasing number of deaths on Georgian roadways. From 2013 to 2016, deaths increased from 514 to 602, a 17% increase. In 2016, 6,939 accidents took place, resulting in 602 fatalities and 9,951 injuries. At a per capita rate, this amounts to approximately 162 fatalities per million citizens, over three times the EU average in 2015, the latest year in which data is available.

While estimates do not exist for Georgia for how many accidents are caused by distracted and other dangerous driving practices, they are very likely contribute to the high fatality rates on the roads in Georgia. In the study we carried out, 96% of minibus drivers in the control group observations engaged in some form of dangerous or distracted driving.

When it comes to distracted driving, studies from other contexts suggest that a distracted driver’s chances of being in an accident are four times higher. Cell phone use is associated with increased incidence of accidents among both novice and experienced drivers. Cell phones aside, other distractions such as smoking can increase the risk of traffic accidents. Importantly, commercial vehicle drivers are no exception, with increased risk of accident associated with distracted driving among commercial drivers as well.

While we know that distracted and other dangerous driving practices among Georgia’s minibus drivers are both common and likely to be leading to fatalities in Georgia, feasible and effective policies are needed to reduce the scale of traffic accidents on Georgia's roadways. In the coming posts, we put forward a number of evidence based policy recommendations that could help do just that.

*Note: This blog post previously erroneously stated that Georgia had among the worst traffic death rates in the world, based on another article. After checking a primary data source, this statistic likely refers to the fact that Georgia has among the highest death rate in wider Europe.

Roads and Safety in the South Caucasus

According to the World Health Organization (WHO), every year about 1.3 million people die as a result of road accidents worldwide. In 2011, the UN launched the Decade of Action for Road Safety 2011-2020. A year later, the UN General Assembly adopted Resolution A/66/L.43 to improve road safety conditions worldwide. This blog addresses satisfaction with road conditions and road safety in the former Soviet Union, specifically in the South Caucasus region.

GALLUP surveys conducted in 2011 show that only 36% of the adult population in the former Soviet Union is satisfied with the roads and highways in areas where they live.  In part, the surveys focused on the South Caucasus, Central Asia, Russia, Moldova, the Ukraine, and Belarus. Satisfaction with roads is highest in Georgia (69%) and lowest in Moldova (17%). In the South Caucasus, Georgia is followed by Azerbaijan (51%) and Armenia (37%). 

In general, unsafe vehicles, poor road infrastructure, careless driving habits, drunk-driving, and speeding are major causes of traffic accidents in the region.  Based on the WHO European Status Report on Road Safety, mortality rates from traffic injuries in the Commonwealth of Independent States (CIS) have increased since the late 1980s compared to the EU. However, it is also important to note that there are differences within the EU region as well.
  

From the late 2000s, Azerbaijan, Armenia, and Georgia have adopted road safety legislature and promoted safety campaigns, such as the 2010 Road Safety Program Project in Azerbaijan, the 2009 National Road Safety Strategy for Armenia and Yerevan and Five Year Action Plan, and road safety enforcement measures and traffic policing in Georgia. These campaigns can have a positive impact on society. Improving roads and highways in the region might contribute to the fact that, according to the WHO European Health for All Database (HFA-DB), death rates for road traffic accidents have been generally decreasing since the late 90s in Armenia, Azerbaijan, and Georgia.

Compared to the 1990s, the standard of roads has improved in the South Caucasus. However, careless driving practices are still common, and it takes much longer to change driving culture. Overall, traffic injuries can continue to be prevented by improving road infrastructure, enforcing speed limits, preventing driving while intoxicated, and promoting more road safety campaigns in the region.

Road Safety and the South Caucasus | WHO Global Status Report on Road Safety

A recent report published by the World Health Organization (WHO) claims that over 90% of the world’s fatalities on the roads occur in low-income and middle income countries, which constitute only 48% of the world’s vehicles. The study assesses the status of road safety in 178 countries, identifies gaps in road safety and proposes recommendations for intervention.

A principal finding reveals that very few countries have comprehensive road safety laws that are effectively enforced. Hence, only 29% of the countries meet basic criteria for reducing speed in urban areas, and less than 10% of countries rate the enforcement of their speed limits as effective. Moreover, only 20% of low-income countries have a law requiring young children in cars to be in car restraints in contrast to 90% of high income countries , which have similar regulations. Additionally, only 57% of countries require seatbelts to be used by passengers in both front and rear seats.

The report provides data on the three South Caucasus countries as well: the separate country profiles include data on the availability and enforcement of road safety laws, statistics on the vehicles and road traffic fatalities. According to the study Georgia has the highest number of fatal and non-fatal traffic accidents in the Caucasus. The data from the Ministry of Internal Affairs shows that 737 road traffic fatalities and 7,349 non-fatal road traffic injuries were reported in 2007. Officially in 2007 the number of the reported road causalities was lower in Armenia: 371 reported fatalities and 2,720 non fatal road traffic injuries, with a significant majority of incidents involving pedestrians. However, the actual numbers in both countries could possibly be higher, considering the fact that in some cases the non-fatal injuries go unreported and post-accident results are rarely recorded.

Yet, Georgia scores surpisingly high on the enforcement of the road safety laws. -- perhaps given its quite successful police reform and recent emphasis on breathalyzer tests. Drunk-driving law enforcement is rated nine out of ten (ten being the most effective), though the State Road Police 2007 data reveals that 37% of the road traffic deaths in the country involved alcohol. Road safety law enforcement scores are striking in Azerbaijan as well, where all the road traffic law enforcements, including drunk-driving, motorcycle helmet, seat-belts and child restraints are rated nine out of ten.

When drawing conclusions, however, we need to take into consideration that the ratings are based on the results of a self-administered survey, and represent the populations’ perceptions of the law enforcement, rather than the facts. While the report provides information on the methodology, it is still unclear, for example, how many people were surveyed and how the respondents were selected.

The full report can be found here.