What video games could learn from social networks about fighting online disinformation

Politics are finding a home in online video game communities, and researcher Daniel Kelley says game companies should begin looking to lessons from social networks about how to combat disinformation.

While there's little evidence to suggest misleading content is rampant in these online communities, Kelley — assistant director of the U.S.-based Anti-Defamation League's Center for Technology and Society — says that game companies' terms of use may not be adequately prepared for what could come.

Kelley's comments, published earlier this month in Slate, come on the heels of a push by the campaign of U.S. Democratic presidential nominee Joe Biden that made digital campaign lawn signs available for supporters in the hugely popular Nintendo game, Animal Crossing: New Horizons.

The signs offer players the ability to show support for Biden and his vice-presidential running mate Kamala Harris when friends and other players visit their virtual island. Day 6 reached out to Nintendo of Canada, but they did not provide a comment.

Here is part of Kelley's conversation with Day 6 host Brent Bambury.

Animal Crossing players might find the idea of a Biden-Harris lawn sign appearing on their island as being a little bit lame, but otherwise this seems innocuous enough. What alarm bells do they raise for you?

In the present moment, it may be innocuous, but I think the idea is that it opens up a question for how prepared video game companies like Nintendo are to deal with their spaces becoming more political. 

We saw earlier this year [U.S.] Representative Alexandria Ocasio-Cortez entering and visiting people's islands in Animal Crossing. Over the last several months, we've seen protests for animal rights, vigils for Black Lives Matter. 

So it's unclear to me, and I think people who think about content moderation issues in digital social spaces, the degree to which Nintendo and other large video game companies are prepared to really address the darker side of political participation in their spaces.

And you can't underestimate the inventiveness of players when it comes to subverting some of this stuff. For example, these lawn signs, can you describe how they've been abused by some players?

I don't know that we've seen, necessarily, the abuse of these specific lawn signs, but I think we can imagine how they would be abused. 

Imagine that someone takes these individual lawn signs and arranges them in their digital space in the form of a swastika. I imagine that's possible.... If you look at the current policies around hateful content, disinformation in Animal Crossing, they're extremely vague and limited and are reminiscent of social media in their early days, maybe circa 2006.

I think we need to take a long arching view of where these spaces could go and prepare.- Daniel Kelley, director of the Anti-Defamation League Center for Technology and Society 

Where else are we seeing politics emerge, maybe unexpectedly, in multiplayer games?

One example that I flag is there was a conversation about race that happened in Fortnite, in wake of the murder of George Floyd by law enforcement in the U.S. 

What happened was there was a commentator, Van Jones, who was part of a panel on race that was streamed inside of Fortnite. And in the space where it was streamed, [players] have the ability to throw tomatoes at each other. And so [some] the players started to throw tomatoes at the image of Van Jones talking about race.

Obviously it was not what Fortnite intended for the event, but I think in terms of their ability to moderate that space, it raises really interesting and potentially two really challenging questions. 

Van Jones is a Black man and they are talking about race, and so he may have been targeted because of his race and it may have been a result of racism. On the other hand, Van Jones has been criticized for his support of the Trump administration's plans around criminal justice, and so it could very easily be that the tomato throwing in that instance was a form of protest against his involvement there. 

How does Fortnite, and Epic Games, their creator, as a company differentiate between the motivations in that case? This is something that is an issue in traditional social media, but I think when we have this added element of behaviour and avatars behaving in different ways in game spaces, it becomes a really challenging content moderation problem.

We know Facebook, for all of its problems and all of its inability to take on some of these issues, has evolved in terms of creating policies, and they have quite complex policies now. Is that conversation going on amongst the manufacturers of games?

I think it's starting. There are groups like the Fairplay Alliance, which is a coalition of over 100 different game companies that is trying to foster these kinds of conversations. I think there are definitely internal advocates inside companies that are trying to push companies to do better in these areas. 

But I think we have a case study and a history in looking at social media, right? We have the ability to look back on 10 years or more of social media history and say, "OK, what can we learn from the mistakes and the problems that social media has had?" 

It goes without saying that online multiplayer games were among the first digital spaces before social media. I grew up playing Everquest and Ultima Online, which were digital social spaces before I joined Facebook and many others. But I think the scale and the complexity of the problems that social media has faced are definitely an area where the game industry needs to look to and adapt — and to do so quickly.

Going back to the political signs in Animal Crossing, we have to wonder: with the U.S. federal election — this very contentious election — just over a month away, do you think it's possible that video games will be used to spread misinformation that affects this year's vote? 

Do you think that there are people trying to figure that out right now?

I don't know, and I don't think we do know. I think what we can do and what I'm trying to do through conversations like this, through articles, through conversation with companies, is to encourage companies to prepare for that happening. 

We've seen the same patterns of sort of transgressive behaviour occur in each generation and iteration of social spaces. So I don't know that we can necessarily today point to the data saying, "Oh, yes, disinformation is definitely happening in Animal Crossing or in Fortnite." 

But I think what we can say is in 2006, when Facebook first opened up to beyond college students, I don't think we could imagine in 10 years it would be the focal point of a discussion around the 2016 election. Or that in 2017 it would be implicated in genocide.

I think we need to take a long arching view of where these spaces could go, and prepare.

Written by Jason Vermes. Produced by Annie Bender. This Q&A has been edited for length and clarity.