The impact of racism in social media

Promotion of racist & inflammatory content

Facebook & Youtube’s algorithms promote extreme content, and you might have wondered why this happens.

There are certain behaviours that Social Media sites are optimised for. These behaviours drive metrics, which in turn drive revenue. For example, Instagram users instinctively check the number of Likes in their posts frequently (1).

This behaviour is not coincidental – the platform has thought of ways to encourage it. The more Likes someone gets, the more content they are likely to post, and the more time they and their followers are likely to spend on Instagram.

For the Social Media site, this leads to more opportunities to learn what kind of content and products their users like and therefore more opportunities to advertise relevant content with a higher probability of interest or purchase.

The focus is on advertising

Instagram, Facebook and YouTube sell advertising, so it is in their best interest to design their apps to encourage certain behaviours. The Like button has a lot of influence on revenue. Some other revenue-driving metrics include:

• How long you spend viewing content
• Engaging with content (likes, retweets, comments)
• Sharing content

Inflammatory content tends to drive engagement and therefore drive up these metrics.

Algorithms

Algorithms are adjusted to drive more engagement and therefore content that is inflammatory including racist, sexist or other prejudiced content is disseminated quickly and can have far-reaching consequences, influencing the behaviour and views of everyday people.

This is exacerbated by the increasing use of bots (2).

The influence of social data

In 2016, Microsoft created an AI bot on Twitter called Tay (@TayandYou on Twitter) (3). As we learnt earlier, machine learning is a process where the computer system learns from making inferences from data inputs. In this case, the data inputs came from Twitter users who would interact with the bot.

The intention was for Tay to become smarter through interactions it had with Twitter users. Microsoft said, “Tay is an artificially intelligent chatbot developed by Microsoft’s Technology and Research and Bing teams to experiment with and conduct research on conversational understanding.”

Within 24 hours, Tay had been taken offline as it had started tweeting racist, sexist and homophobic comments. This happened because some Twitter users sent racist, sexist and homophobic tweets to the bot (4). Tay had learnt this toxic ‘behaviour’ from its interactions with humans, which demonstrated how AI can amplify the worst of human behaviour.

References:

1. Janarthanan Balakrishnan & Mark D. Griffiths, 2019. An exploratory study of “selfitis” and the development of the selfitis behaviour scale, NCBI.
2. Massimo Stella, Emilio Ferrara & Manlio De Domenico, 2018. Bots increase exposure to negative and inflammatory content in online social systems, PNAS.
3. Peter Lee, 2016. Learning from Tay’s introduction – The Official Microsoft Blog, Microsoft.
4. Oscar Schwartz, 2019.In 2016, Microsoft’s racist chatbot revealed the dangers of online conversation, IEEE Spectrum.

Further resources:

1. Alastair Reed, Joe Whittaker, Fabio Votta and Seán Looney, 2019. Radical Filter Bubbles, Royal United Services Institute for Defence and Security Studies.