Johnsons Team schlägt vier Optionen vor. Zum einen sei es effektiver und einfacher, systematisch kleine und kleinste Hass-Zellen zu bekämpfen, um die Bildung größerer Gruppen zu unterbinden. Außerdem sollten statt ganzer Gruppen zufällig ausgewählte Mitglieder verschiedener Gruppen von den Plattformen verbannt werden, um eine Reorganisation des Netzwerks zu vermeiden.

Zwei weitere Strategien basieren darauf, die Netzwerke durch andere, gegnerische Gruppen von Nutzerinnen und Nutzern angreifen zu lassen: Einerseits sollten demnach den Extremisten feindlich gesinnte Gruppen vom Netzwerk gefördert werden – andererseits schlägt Johnsons Team vor, Extremisten mit unterschiedlichen Ansichten aufeinanderzuhetzen.

Interconnected hate clusters form global ‘hate highways’ that—assisted by collective online adaptations—cross social media platforms, sometimes using ‘back doors’ even after being banned, as well as jumping between countries, continents and languages. Our mathematical model predicts that policing within a single platform (such as Facebook) can make matters worse, and will eventually generate global ‘dark pools’ in which online hate will flourish. We observe the current hate network rapidly rewiring and self-repairing at the micro level when attacked, in a way that mimics the formation of covalent bonds in chemistry. This understanding enables us to propose a policy matrix that can help to defeat online hate, classified by the preferred (or legally allowed) granularity of the intervention and top-down versus bottom-up nature.

The most effective disinformation has always been that which has a kernel of truth to it, and indeed most of the content being disseminated now is not fake—it is misleading. Instead of wholly fabricated stories, influence agents are reframing genuine content and using hyperbolic headlines. The strategy involves connecting genuine content with polarizing topics or people. Because bad actors are always one step (or many steps) ahead of platform moderation, they are relabeling emotive disinformation as satire so that it will not get picked up by fact-checking processes. In these efforts, context, rather than content, is being weaponized. The result is intentional chaos.

A collection of several hundred late-2000s internet memes posted to image sharing site Imgur, and subsequently linked to on Reddit, provides a perfect example (“Late 2000s imagedump (352 images),” 2019; “Late 2000s imagedump,” 2019). Appropriately, the first comment reads, “A more simple time.” Ha ha ha, here’s a grainy photo of two photoshopped cows. “Moo,” one cow’s dialogue box says. “You bastard, I was going to say that!” says the second. Here’s a guy with his mouth photoshopped over both his eyes. Here’s two cats photoshopped to look like they’re playing a handheld videogame console. “LET ME SHOW YOU MY POKEMONS!” the cat says. This image is ensconced in an additional text frame, which at the top reads “Pokemons,” and at the bottom, “Let me show you them.” Here’s a bear running onto a golf course. The top caption indicates that this is golf course rule enforcement bear. The bottom caption concludes that you are fucked now. Here’s some kittens making sassy faces at each other.

Like I said: a simpler time, but oh right, also, the thread begins with a lighthearted meme about Hitler. And continues with dehumanizing mockery of a child with disabilities. And more sneering mockery of an old man hooked up to an oxygen tank. And date rape. And violence against animals. And fat shaming. And homophobia. And racism. And pedophilia. And how hilarious 9/11 was. And women as unfeeling, inanimate sex objects. With multiple examples of the last seven.

If this were a collection, specifically, of 4chan memes, that flagged itself as representing early trolling subculture, many would nod and say, yes, they really were pieces of shit back then. Those trolls! They helped elect Donald Trump you know.

But this was not a collection, specifically, of 4chan memes. This was a sampling—and as a person who has studied such things for the better part of a decade, I can attest that it is a representative sampling—of what was often described as “internet culture,” or simply “meme culture,” from about 2008 to 2012. Such a broad framing belies the fact that internet/meme culture was a discursive category, one that aligned with and reproduced the norms of whiteness, maleness, middle-classness, and the various tech/geek interests stereotypically associated with middle-class white dudes.2 In other words: this wasn’t internet culture in the infrastructural sense, that is, anything created on or circulated through the networks of networks that constitute the thing we call The Internet. Nor was it meme culture in the broad contemporary sense, which, as articulated by An Xaio Mina (2019), refers to processes of collaborative creation, regardless of the specific objects that are created. This was a particular culture of a particular demographic, who universalized their experiences on the internet as the internet, and their memes as what memes were.

Johnsons Team schlägt vier Optionen vor. Zum einen sei es effektiver und einfacher, systematisch kleine und kleinste Hass-Zellen zu bekämpfen, um die Bildung größerer Gruppen zu unterbinden. Außerdem sollten statt ganzer Gruppen zufällig ausgewählte Mitglieder verschiedener Gruppen von den Plattformen verbannt werden, um eine Reorganisation des Netzwerks zu vermeiden.

Zwei weitere Strategien basieren darauf, die Netzwerke durch andere, gegnerische Gruppen von Nutzerinnen und Nutzern angreifen zu lassen: Einerseits sollten demnach den Extremisten feindlich gesinnte Gruppen vom Netzwerk gefördert werden – andererseits schlägt Johnsons Team vor, Extremisten mit unterschiedlichen Ansichten aufeinanderzuhetzen.

Interconnected hate clusters form global ‘hate highways’ that—assisted by collective online adaptations—cross social media platforms, sometimes using ‘back doors’ even after being banned, as well as jumping between countries, continents and languages. Our mathematical model predicts that policing within a single platform (such as Facebook) can make matters worse, and will eventually generate global ‘dark pools’ in which online hate will flourish. We observe the current hate network rapidly rewiring and self-repairing at the micro level when attacked, in a way that mimics the formation of covalent bonds in chemistry. This understanding enables us to propose a policy matrix that can help to defeat online hate, classified by the preferred (or legally allowed) granularity of the intervention and top-down versus bottom-up nature.

The most effective disinformation has always been that which has a kernel of truth to it, and indeed most of the content being disseminated now is not fake—it is misleading. Instead of wholly fabricated stories, influence agents are reframing genuine content and using hyperbolic headlines. The strategy involves connecting genuine content with polarizing topics or people. Because bad actors are always one step (or many steps) ahead of platform moderation, they are relabeling emotive disinformation as satire so that it will not get picked up by fact-checking processes. In these efforts, context, rather than content, is being weaponized. The result is intentional chaos.

A collection of several hundred late-2000s internet memes posted to image sharing site Imgur, and subsequently linked to on Reddit, provides a perfect example (“Late 2000s imagedump (352 images),” 2019; “Late 2000s imagedump,” 2019). Appropriately, the first comment reads, “A more simple time.” Ha ha ha, here’s a grainy photo of two photoshopped cows. “Moo,” one cow’s dialogue box says. “You bastard, I was going to say that!” says the second. Here’s a guy with his mouth photoshopped over both his eyes. Here’s two cats photoshopped to look like they’re playing a handheld videogame console. “LET ME SHOW YOU MY POKEMONS!” the cat says. This image is ensconced in an additional text frame, which at the top reads “Pokemons,” and at the bottom, “Let me show you them.” Here’s a bear running onto a golf course. The top caption indicates that this is golf course rule enforcement bear. The bottom caption concludes that you are fucked now. Here’s some kittens making sassy faces at each other.

Like I said: a simpler time, but oh right, also, the thread begins with a lighthearted meme about Hitler. And continues with dehumanizing mockery of a child with disabilities. And more sneering mockery of an old man hooked up to an oxygen tank. And date rape. And violence against animals. And fat shaming. And homophobia. And racism. And pedophilia. And how hilarious 9/11 was. And women as unfeeling, inanimate sex objects. With multiple examples of the last seven.

If this were a collection, specifically, of 4chan memes, that flagged itself as representing early trolling subculture, many would nod and say, yes, they really were pieces of shit back then. Those trolls! They helped elect Donald Trump you know.

But this was not a collection, specifically, of 4chan memes. This was a sampling—and as a person who has studied such things for the better part of a decade, I can attest that it is a representative sampling—of what was often described as “internet culture,” or simply “meme culture,” from about 2008 to 2012. Such a broad framing belies the fact that internet/meme culture was a discursive category, one that aligned with and reproduced the norms of whiteness, maleness, middle-classness, and the various tech/geek interests stereotypically associated with middle-class white dudes.2 In other words: this wasn’t internet culture in the infrastructural sense, that is, anything created on or circulated through the networks of networks that constitute the thing we call The Internet. Nor was it meme culture in the broad contemporary sense, which, as articulated by An Xaio Mina (2019), refers to processes of collaborative creation, regardless of the specific objects that are created. This was a particular culture of a particular demographic, who universalized their experiences on the internet as the internet, and their memes as what memes were.