The survey website Numbeo is frequently cited by many media outlets in articles ranking, amongst other things, the safest and most dangerous countries and cities in Europe. But its crowdsourced methodology is unreliable.
Picture yourself throwing a dart on a map of Europe to choose your destination for your sabbatical year (you’re a very adventurous person). Your dart lands in Sweden. What’s the first thing you do? If you’re like me, you’d probably look some things up on Google or Bing. A query like “Sweden cost of living” would probably take you to Numbeo. A couple more clicks on Numbeo would take you to the website's “Crime index by country”, according to which Sweden is the most dangerous country in Europe. The index table appeared on a viral Facebook post (in Swedish) that was shared more than a thousand times. But the sources to back this claim appear to be, to say the least, unscientific. Our Swedish partner Viralgranskaren took a look at it.
Numbeo, a Serbian website, bases its index off of self-reports from anyone on the Internet. Viralgranskaren could, without having to log into the website, rank Oslo and Norway as insecure (even though they are based in Stockholm, Sweden). Their survey directly affected the average results for Oslo as they are based on 680 respondents only. Even worse, the Crime index rates Estonia as the safest country in Europe in 2019. But the “crime data” for Estonia comes from a survey of 194 visitors, some of which could be from anywhere in the world.
“The reports are very weak because there’s so little data”, said Torbjörn Sjöström, CEO of the research firm Novus, to Viralgranskaren.
That does not keep the Numbeo rankings from spreading around Europe, whether it pertains to the cost of living, the quality of life, or the safety of a city.
Bogus rankings becoming really viral
Viralgranskaren had already reported on Numbeo in January 2017, as the website’s “crime index” was used by various outlets including conspiracy website Infowars, with an article headlined “CULTURALLY ENRICHED MALMO, SWEDEN IS THE MOST DANGEROUS CITY IN WESTERN EUROPE”. The same story could be found, for instance, in the Danish tabloid B.T. Except that one single man, Linus Trulsson, was later able to tweak the ranking to make Lund, a neighboring city, the most dangerous city in the world, right in front of Caracas, Venezuela. That being said, Numbeo has some level of protection against spam, and FactcheckEU tried to instill fake data into the website, some of our attempts failed.
Because rankings often go viral, statistics from Numbeo are frequently used by European media, mainstream or not. More than a year after the Malmö story, French news network BFMTV published its own report: “According to an American (sic) poll, Marseille is the most dangerous city in Europe”. The official Facebook post for the article was shared 5000 times. Even if the article mentioned the ranking’s dubious methodology, the post was captioned: “People of Marseille are European champions”. Just last month, Euro Weekly News, an English-language newspaper in Spain, published a column criticising UK tabloids while citing Numbeo’s “crime index” in its first paragraph. The Financial Times used the website’s data for a story on gentrification in Europe.
Numbeo’s crime index is successful in part because it’s very hard to compare crime statistics from country to country, especially when it comes to sexual crimes. In its last report on crime in Europe, Eurostat explained that “in Instances of police-recorded sexual violence relative to population size varied greatly in 2016 – between 5 and 190 per 100 000 people. This huge variation is probably affected both by differences in what is perceived as a crime and by the proportion of crimes reported to the police.” One good exemple of that is Sweden’s numbers on sexual offences and rapes, which were higher than all other European countries but England and Wales in 2016. Sweden has a broader definition of what is a sexual offence or a rape, as well as a specific way of counting crimes.
The Deutsche Bank used Numbeo as well
It’s not just crime data: the German website of the Russian government-owned network Sputnik published Numbeo’s ranking of the “dirtiest cities in Europe” in 2018.
But outlets are mostly using Numbeo’s cost of living index to fill up articles. This tool is likely to be more reliable for tourists or prospective immigrants because responding to a survey about the price of a litre of milk is different than responding to questions about how safe you feel where you live. Still, anyone can change the data for any city and this part of the methodology is rarely explained in articles. In Italy, la Reppublica, il Sole 24 Ore, or La Stampa have been using Numbeo. FactcheckEU partners have also found exemples of articles based on Numbeo in Germany, Belgium, Croatia, Denmark or in Lithuania. The Deutsche Bank, the largest bank in Germany, even used Numbeo’s figures (including on crime or pollution) for its own quality of life index in 2017. This research was itself shared by… the official Danish statistics office.
Jules Darmanin with Johan Wikén, Sanja Despot, Silvia Cavasola, Derek Thomson, Cristina Helberg, Liepa Želnienė, Maarten Schenk, J.P. Burger.