Safety survey formula skews SA ranking

There has been a rise in global statistical initiatives that measure and rank countries in terms of various aspects of the human condition. Some of the more prominent examples include the Human Development Index, the World Governance Indicators, the Global Peace Index and the Corruption Perceptions Index.

Each ranks countries according to a series of indicators, or a composite indicator, and tracks their progress or decline over time.

One of the most recent global indicator projects is the Gallup Law and Order Index. It ranks 142 countries based on a perception survey relating to personal safety and policing, from a representative sample of 1000 people in each country. Knowing how secure, or insecure people feel is important because insecurity affects economic growth and undermines development.

According to the recently released 2018 law and order index, South Africa ranks high in the insecurity index – 137 out 142 countries. This means that South Africans would have expressed high levels of insecurity as well as fear that they were likely to, or had already, fallen victim to crime.

The ranking suggests that South Africans consider themselves to be more insecure, and having lower levels of confidence in the police, compared to people in Yemen, the Democratic Republic of Congo (DRC), the Central African Republic (CAR), Libya and Mali. These are all unstable states affected by violent conflict and high levels of instability.

This is surprising given that South Africa is not in state of armed conflict and is relatively stable. The possible reason for such a questionable ranking is that the survey, like many global perception surveys, doesn’t adequately account for the extent to which people will provide unreliable information about sensitive issues. To improve accuracy, surveys like this should factor in differences in context.

The rankings

The rankings are based on an index score derived from responses to the following questions:

• In the city or area where you live, do you have confidence in the local police force?

• Do you feel safe walking alone at night in the city or area where you live?

• Within the last 12 months, have you had money or property stolen from you or another household member?

• Within the past 12 months, have you been assaulted or mugged?

It’s undeniable that South Africa has high levels of insecurity and interpersonal crime. And, there’s a significant trust deficit between citizens and the police.

For example, the StatsSA 2016/17 Victims of Crime Survey showed that only 30% of South African’s reported feeling safe walking at night in their neighbourhoods. Only 57% of households reported that they were “satisfied” with the police in their communities. And the country has very high levels of crime.

Nevertheless, it seems odd that South Africa is ranked below countries like Yemen, which has been in the throes of an intense civil war for several years, the Central African Republic and Libya, which have been acutely affected by insurgency, criminality and weak law and order institutions.

Ranking South Africa below the DRC, Mali and Libya is also questionable given that the security forces and militias in those countries have been widely regarded as predatory and highly abusive.

So what’s missing?

Context

Firstly, context is key.

A key shortcoming of using survey data about crime and insecurity to construct indices and rankings is that people won’t always reply to questions honestly and accurately.

In stable democracies respondents will often give precise and truthful responses as there is little or no fear of reprisals from the state. Conversely, in unstable countries that have repressive governments, research shows that citizens are less willing to provide accurate information about personal experiences of crime and policing. This is because they fear there may be negative repercussions for them and their families.

Secondly, as research method experts have argued, survey responses can also be influenced by a variety of societal norms, particularly those related to privacy and dignity, in which sensitive matters aren’t easily discussed with strangers.

In South Africa, citizens are generally willing to talk openly about crime and to criticise the police. But, this isn’t the case in many other African and Latin American countries that were rated as being safer. These include DRC, Libya, Honduras and Mexico.

This is not to say that constructing indices about crime victimisation and policing on a country basis is irrelevant. But the danger of indicators like this, and adopting a ranking approach without careful consideration of the context in which the data is gathered, is that it could lead to wrong perceptions about crime and policing. That may even reinforce the use of militarised policing strategies, which will further undermine human security over less aggressive and more integrated approaches to crime prevention. Examples of where this has happened include Brazil, Mexico and South Africa.

To improve the accuracy of indices like this, it would be advisable to develop a quantifiable weighting for the reliability of crime and insecurity survey data for each country, and then apply the weighting to the overall index score. For example, in countries with more authoritarian governments, respondents are likely to under report their levels of trust in the police and sense of personal insecurity.

Applying a reliability weighting would adjust the overall insecurity index score to better reflect people’s lived reality. Such a weighting can be developed by including additional questions in the survey, for example about how willing respondents are to talk to strangers about sensitive information, including views about their governments.

Guy Lamb, Director, Safety and Violence Initiaitive, University of Cape Town.