Bartholomew's Notes

From the website of Evolution AI, a London-based data science consultancy:

In a report published by the Hope Not Hate campaign last Monday, researchers Dr Imran Awan and Dr Irene Zempi describe widespread celebration of the murder of Labour MP Jo Cox on the Twitter website during the months of June and July of 2016.

The findings of the report do not appear to stand up to fact-checking… We could find evidence of less than seventy tweets in support of Jo Cox’s murder. This implies that the claims are inflated by a factor of around one thousand.

…The Hope Not Hate report, Jo Cox ‘deserved to die’: Cyber Hate Speech Unleashed on Twitter, seems to be a conflation of two analyses, one concerning brexit-related tweets and another concerning tweets about Jo Cox. It is not carefully written and it is often not clear which data set is being referred to. Our analysis suggests that the 50,000 tweet claim belongs to tweets about ‘brexit’ while a vastly smaller dataset is actually about Jo Cox…

Although it seems that there are severe methodological errors in the Hope Not Hate report, we do not suggest that the authors have been deliberately misleading. We have informed the authors that we cannot reproduce their results and have asked them to share their dataset or clarify their methods. They told us they are currently unable to do so.

Cox’s murderer was convicted last month. There is, of course, a natural tendency for any campaign against a social evil to exaggerate the extent of it – not necessarily cynically, to raise funds or gain influence, but also perhaps due to confirmation bias or the perceptual effect of dealing with the issue on a daily basis (I recently noted similar problems with statistics and claims being bandied about by an anti-stalking organisation). There may be a temptation to rationalize this, saying “well, it’s better that people worry too much than not enough”, but as the Evolution AI authors note,

This study risks normalizing extremely rare viewpoints and representing them as more common and more acceptable than they truly are. This will have precisely the opposite effect to that intended. In general, ‘fake news’ in respected media sources acts to decrease public trust and should not go unchallenged.

Evolution AI’s work is discussed further and expanded by The Economist, which adds the detail that

When The Economist asked the [Hope Not Hate] authors for help, they declined to share their data with us, citing death threats they said they had received since the report’s release. 

The Hope Not Hate authors referred to these threats on the day of their report’s publication; Zempi wrote on Twitter that

Sorry to say that both @ImranELSS & I have received death threats because of our new study on #Cyber #HateSpeech BUT we will not be silenced

Given this commendably robust attitude, it seems very odd that the authors should now be intimidated from producing the crucial data that would reassure the public of their study’s integrity. It is not at all clear why doing so would constitute a greater or renewed threat to their personal safety; and Awan has reportedly been a target of the far-right for the past two years anyway, since publishing a report on online Islamophobia.

To compound the problem, the specific claim that 50,000 Tweets celebated Cox’s death or praised her killer comes from a  Hope Not Hate press release rather than the report itself. Hope Not Hate has acknowledged that this was an error, but the story has has been pulled from the Guardian, which now carries a notice that

This article has been removed. It was based on a press release from anti-racism campaigners Hope Not Hate which it admits contained incorrect information.

UPDATE: It has now come to my attention that the report authors made a further statement two days ago:

This was a qualitative study (analysis of a snapshot of views) rather than a quantitative study, which ’number crunches’ data to produce an empirical analysis.

…One of the themes we identified in our sample was the claim that Jo Cox had ‘deserved to die’ because she supposedly supported so-called ‘rape gangs’, and had been a ‘traitor’ who ‘got what she deserved’.

As far as the second part of our report highlights, looking at cyber hate responses to Brexit, we pointed out that experiences of xenophobic hostility led to communities feeling a sense of fear, insecurity and vulnerability. We also noted how social media was used to report offline incidents of hate.

This means that Hope Not Hate apparently misrepresented the study as being a calculation of the amount of online hate that was directed at Cox after her death, rather than as being a qualitative study of the various rhetorical strategies that were deployed by those posting hateful Tweets. However, as The Economist notes, the authors were happy enough to endorse misleading media reports that followed the publication. A commentator on the Hope Not Hate website also complains that requests for clarifications on Twitter were met with the “block” button.

Further, the report very clearly makes quantitative claims. For example, in the executive summary we read that:

This study examined over 53,000 tweets between June 2016 and July 2016, following the murder of British MP Jo Cox and the EU Referendum vote.

…A key theme that emerged on Twitter was the depiction of Thomas Mair as a ‘hero’ for murdering Cox. Individuals had tagged pictures in their tweets praising Mair for killing Jo Cox, using the hashtag #HeroMair

Currently, there is just one Twitter result that uses this hashtag (leaving aside a few new ones pointing out this fact), and this particular item is cited in the report. A Google Search brings up evidence of a second example from June, from an account that has since been suspended, but even allowing for suspensions and deletions one would expect a few more examples to be left over on Twitter or preserved on Google search results. One might also have expected a few responses from people referring to the existence of the hashtag in order to express their disgust with it – but there is nothing. Yet the executive summary implies there were many, and one media report (in the Daily Star) interprets this to mean that “the hashtag #HeroMair trended on Twitter for a short time” – a detail I would expect someone to have noticed at the time.

Filed under: Uncategorized