�鶹��

Amy-Louise Watkin and Joe Whittaker look at the interface between terrorism and the internet, and whether big tech companies should be doing more to to tackle online propaganda and the rise in digital radicalisation

At first glance, the removal of terrorist content online seems like an intuitive goal. In the heyday of the Islamic State’s virtual caliphate, the group was able to spread their message far and wide. Scholars found that online platforms were being used to spread propagandaⁱ, recruit potential terroristsⁱⁱ, and disseminate instructional materialⁱⁱⁱ. Far right terrorists, too, have abused platforms by disseminating content. The world watched in horror as the Christchurch attack was live-streamed on Facebook, and multiple attackers have posted their manifestos online^iv. Although different countries have different freedom of speech norms enshrined in law, there has been a widespread move towards removing terror content from the Internet, including the Christchurch Call^v, the UK Online Harms White Paper^vi, and the German NetzDG law^vii.

Although they are sometimes maligned for not acting quickly enough, the big tech companies have, by and large, adopted policies which help to remove terror content from their platforms, as well as working together to share best practices, like in the case of the Global Internet Forum to Counter Terrorism^viii and Tech Against Terrorism^ix. There have been considerable successes to this approach; many have documented the stark difference between the reach of groups like ISIS before tech companies took a more robust approach to content removal compared to after^x. It is clear that the degradation of the group’s online presence was, at least in part, caused by tech companies taking a proactive approach, however, the issue contains many complexities. Below we offer a number of considerations for the removal of content, including trade-offs, regulatory frameworks, and the capabilities and motivations of social media platforms.

The online extremist’s dilemma
Although the removal of terrorist content has clear benefits, there may be significant trade-offs too. The move of Islamist groups like ISIS away from the larger platforms towards smaller and more secure ones, particularly Telegram, has been well documented^xi. The unique selling point of these platforms is their operational security. As such, turning terrorist actors away from mainstream platforms may inadvertently be helping them. Research has suggested that terrorists’ use of the Internet (including using social media to communicate with co-ideologues, accessing ideological content, and using the web to plan their attack) makes them less likely to succeed, while finding that the use of end-to-end encrypted platforms does not affect success^xii. Similarly, it was also found that terrorists that use social media are more likely to be apprehended before their event than those who did not^xiii. This is what Clifford and Powell call the online extremist’s dilemma; the goals of outreach for recruitment and operational security are fundamentally at odds^xiv. There are clear and substantial benefits to removing terrorist content, but we must be honest about the trade-offs; terrorists have less opportunity to telegraph their actions and ideology to law enforcement.

In addition to the complex trade-offs of content removal, the creation of regulation requires some further considerations^xv. One is the variety of services that terrorists and extremists are using and the finding that their use is interconnected yet not homogeneous. Weirman and Alexander^xvi revealed that ISIS have adapted to the policies of the major platforms. For example, instead of posting clearly violating content on Twitter, they post non-violating news sources that validate the group’s stance. This research along with other studies have shown that the major platforms are also being used to post URL links that redirect followers to a range of file-sharing sites (which tend to be less censored), where large quantities of content (that would violate major platforms policies) are posted^xvii.

Further to this, Clifford and Powell found that ISIS also use the major platforms to redirect to the previously mentioned online instant messaging service Telegram where followers can communicate with one another both in public channels and private group chats, post media and instructional guides^xviii. Therefore, regulation needs to consider that there are a range of social media platforms, file-sharing sites and instant messenger services being used with Tech Against Terrorism finding evidence of terrorist groups using more than 330 platforms with half of the top 50 most-used platforms by ISIS being small or micro-platforms^xix. Regulation also needs to recognise the different ways the platforms are used with some used to post currently non-violating content, others to signpost to other sites, and some as repositories or for communication.

Regulatory strategies
The final consideration will look at the range of capabilities and resources, as well as motivations of the platforms, file-sharing sites and instant messenger services to counter this issue. The major platforms hire thousands of employees globally to work on safety and security^xx, bring in large advertising revenues^xxi, and claim to invest heavily in an array of artificial intelligence technology that proactively searches for and removes terrorist content^xxii, despite the finding that this content can still be easily found on their sites^xxiii. This is not typical, however, of smaller file-sharing sites, alternative platforms, and instant messenger services. Such services often only have a handful of employees, are based in just one country, and rely on strategies other than the selling of advertisements for revenue.

For example, the file-sharing site JustPaste.it does not have the resources and capabilities of larger platforms. However, it works with Tech Against Terrorism^xxiv and the Global Internet Forum to Counter Terrorism^xxv to try to tackle terrorist use of their sites^xxvi. Other platforms and sites are less cooperative with a heavy focus on free speech and user privacy^xxvii. As a result of this variation, some regulatory strategies and penalties may be easily implementable for some companies but create struggles and vulnerabilities for others.

Regulatory strategies are therefore going to have to focus on more than just the major platforms, fast content removal, and content that is typically considered violent or inciting. If regulation focuses on the major platforms – or platforms with a certain size user-base – then they will only be focusing on one small part of the online ecosystem used by terrorist and extremist groups. If regulation focuses on violent and inciteful content then groups will be able to continue to post non-violating content and URLs signposting followers elsewhere. It may also lead to the whack-a-mole effect of groups migrating to sites that are less censored and more difficult for law enforcement to monitor. If regulation focuses on fast removals then companies with an enormous volume of content and/or very few resources may feel pressured to err on the side of caution and make removal errors. Finally, regulation that focuses on fining companies who do not comply risks the major platforms treating them as just another business cost and may be difficult to enforce on companies that are registered outside of the jurisdiction.

Regulation has recently moved in the direction of requiring social media platforms to proactively remove terrorist and extremist content as quickly as possible. However, this article argues that regulation should be acknowledging several considerations. First, regulation needs to include the enormous range of platforms (both large and alternative), , file-sharing sites, and instant messenger services that are  used by terrorist and extremist groups instead of focusing on the major platforms, or platforms with a certain number of users. Secondly, they will need to address that the platforms and sites are not used homogeneously and therefore may require different strategies to one another.

Third, they need to consider the differing levels of resources available to the companies, as well as their motivations to comply, and whether penalties, such as fines are going to have any success in incentivising compliance. Finally, regulation should acknowledge that although content removal has benefits, there are also trade-offs to this. Therefore, a one-size-fits-all approach to regulation is unlikely to be effective; having a range of regulatory strategies to draw from should give more flexibility and allow for a more tailored approach depending on the range of important differing factors that have been raised throughout.

Amy-Louise Watkin is a PhD Candidate at Swansea University and a member of Cyber Threats Research Centre.

Joe Whittaker is a joint PhD Candidate at Swansea University and Leiden University. He is also a research fellow at the International Centre for Counter-Terrorism and a member of the Cyber Threats Research Centre.

References
ⁱJ.M. Berger and Jonathon Morgan, ‘The ISIS Twitter Census: Defining and Describing the Population of ISIS Supporters on Twitter’, The Brookings Project on U.S. Relations with the Islamic World: ANALYSIS PAPER, March.20 (2015), 68.
ⁱⁱSeamus Hughes and Alexander Meleagrou-Hitchens, ‘The Threat to the United States from the Islamic State’s Virtual Entrepreneurs’, CTC Sentinel, 10.3 (2017), 1–9.
ⁱⁱⁱAlastair Reed and Haroro Ingram, ‘Exploring the Role of Instructional Material in AQAP’s Inspire and ISIS’ Rumiyah’, 1st European Counter Terrorism Centre (ECTC) Conference on Online Terrorist Propaganda, 2017.
^ivGraham Macklin, ‘The Christchurch Attacks: Livestream Terror in the Viral Video Age’, CTC Sentinel, 2019, 18–29.
^vThe Christchurch Call, New Zealand Ministry of Foreign Affairs and Trade, .
^viHM Government, Online Harms White Paper (London: The Stationary Office, 2019) .
^viiBBC, ‘Germany Starts Enforcing Hate Speech law’, Jan 1st 2018, .
^viiiAbout, Global Internet Forum To Counter Terrorism, .
^ixAbout, Tech Against Terrorism, .
^xJ.M. Berger and Heather Perez, ‘The Islamic State’s Diminishing Returns on Twitter: How Suspensions Are Limiting the Social Networks of English-Speaking ISIS Supporters’, Occasional Paper, 2016, 1–20 Paper.pdf>; Maura Conway, ‘Violent Extremism and Terrorism Online in 2016: The Year in Review’, Vox Pol, 2016; Maura Conway and others, Disrupting Daesh: Measuring Takedown of Online Terrorist Material and Its Impacts (Dublin, 2017); Daniel Grinnell, Stuart Macdonald and David Mair, The Response Of, and on ,Twitter to the Release of Dabiq Issue 15, 2017.
^xiMia Bloom, Hicham Tiflati and John Horgan, ‘Navigating ISIS’s Preferred Platform: Telegram’, Terrorism and Political Violence, 2017, 1–13 Nico Prucha, ‘IS and the Jihadist Information Highway – Projecting Influence and Religious Identity via Telegram’, Perspectives on Terrorism, 10.6 (2016), 48–58; Bennett Clifford and Helen Powell, ‘Encrypted Extremism: Inside the English-Speaking Islamic State Ecosystem on Telegram’, Program on Extremism, 2019.
^xiiJoe Whittaker, ‘How Content Removal Might Help Terrorists’, Lawfare Blog, 2019 .
^xiiiMichael Jensen and others, ‘The Use of Social Media by United States Extremists’, National Consortium for the Study of Terrorism and Responses to Terrorism, 2018 .
^xixClifford and Powell.
^xvWatkin, A. (2019) Considering the whole ecosystem in regulating terrorist content and hate online. E-International Relations. Accessed 8 October 2019 via
^xviWeirman, S., & Alexander, A. (2018). Hyperlinked Sympathizers: URLs and the Islamic State. Studies in Conflict & Terrorism, 1-19.
^xviiMacdonald, S., Grinnell, D., Kinzel, A., and Lorenzo-Dus, N. (2019) A study of outlinks contained in tweets mentioning ‘Rumiyah’. The Global Research Network on Terrorism and Technology; Conway, M., Khawaja, M., Lakhani, S., Reffin, J., Robertson, A., & Weir, D. (2017). Disrupting Daesh: measuring takedown of online terrorist material and it's impacts; Shehabat, A., & Mitew, T. (2018). Black-boxing the black flag: anonymous sharing platforms and ISIS content distribution tactics. Perspectives on Terrorism, 12(1).
^xviiiClifford, B., and Powell, H. (2019) Encrypted Extremism: Inside the English-speaking Islamic State ecosystem on Telegram. Program on Extremism, The George Washington University.
^xixTech Against Terrorism (2019) ISIS use of smaller platforms and the DWeb to share terrorist content. Tech Against Terrorism. Accessed 30 August 2019 via
^xxPotts, N. (2019) UK �鶹�� Affairs Committee: Hate crime and its violent consequences. House of Commons. Accessed 30 August 2019 via Statista (2019a) Number of Twitter employees from 2008 to 2018. Statista. Accessed 30 August 2019 via
^xxiGesenhues, A. (2019) Facebook ad revenue tops $16.6 billion, driven by Instagram, Stories. Martech Today. Accessed 23 Sept 2019 via
^xxiiBickert, M. (2017) Hard questions: How we counter terrorism. Facebook Newsroom. Accessed 23 Sept 2019 via
^xxiiiHouse of Commons �鶹�� Affairs Committee (24 April 2019) Oral evidence: Hate crime and its violent consequences. Accessed 23 Sept 2019 via
^xxivAbout, Tech Against Terrorism, .
^xxvAbout, Global Internet Forum To Counter Terrorism, .
^xxviTech Against Terrorism (2017) Tech Against Terrorism at Chatham House. Tech Against Terrorism. Accessed October 2017 via Lee, D. (2018) Tech firms hail ‘progress’ on blocking terror. BBC. Accessed 30 August 2019 via
^xxviiFor example, Gab,