A Statistical Analysis of Racism Across 10,790 Replays

The Dataset
The dataset consists of 10,790 replays spanning a period of over 8 years, with the majority of the replays occurring between April 2017 and June 2020.

https://i.imgur.com/8bt1AS5.png

Among these 10,790 games, there were 15,769 unique players. Below is the distribution of players based on the number of games they have played within the dataset. This is an important distinction, a player could have played 1000 games of mafia, but if they only appear in a single replay within the dataset, then they will appear in the "1 game played within dataset" category.

https://i.imgur.com/EsRd4TX.png

Slur Usage by Player - Number of Slurs
In this section the total number of messages a player sent that contain a slur are aggregated for that player across all games in which they appear in the dataset.

For example, a player with the following chat logs would have a score of 3. Even though one of their messages contains two slurs, this is not double-counted to avoid skewing the dataset.

Code:

---

Game 1:
Player: {slur} {slur}
Player: hello world
Player: {slur}

---

Code:

---

Game 2:
Player: {slur}

---

https://i.imgur.com/fRUpOo4.png

Zooming in on just the top 5% of the previous graph:

https://i.imgur.com/5AI0gjJ.png

Slur Usage by Player - Number of Games
An alternative way to analyze the data is to count the number of games in which a given player used a slur.

For example, a player that has played 1000 games in the dataset where they used 0 slurs in 999 of those games, but used a slur 200 times in a single game would appear in the top category in the previous section, but would only appear in the "1 game in which player used a slur" category in this section.

https://i.imgur.com/hig2h03.png

Again, zooming in on the top 5%:

https://i.imgur.com/vvHFW4B.png

Slur Usage by Game
This section maps the distribution of games based on the number of messages containing slurs sent by all players within the game.

The following game would have a score of 4 as there were 4 messages sent that contain slurs.

Code:

---

Player 1: Hello World
Player 2: {slur} {slur}
Player 2: {slur} {slur} {slur}
Player 3: {slur}
Player 4: Hello World
Player 2: {slur}

---

https://i.imgur.com/SJKQjq9.png

Preliminary Observations

• 90.95% of players never used a slur across all of their games
• 96.11% of players never used a slur in more than one game
• The top 1.89% of players based on number of slurs used contribute 89.20% of all slurs used

This raises the question: what would the statistics look like if these repeat offenders were filtered from the dataset?

Slur Usage by Game - Filtering Chronic Offenders
Players that have used a slur in more than 3 games are flagged as "chronic offenders" for the purposes of the filter.

These chronic offenders make up the top 1.89% (denoted as Top 2% in graphics for the sake of brevity) of players in the dataset by number of games in which they used a slur.

Two different filtering methods are used.

Method 1: Ignore all messages sent by players in the top 1.89%
In this method all messages sent by the filtered players are simply ignored.

Method 2: Remove games from the dataset in which a filtered player used a slur
The hypothesis behind this method is that there are knock-on effects from the usage of slurs by these chronic offenders - a chronic offender using slurs in game is likely to increase the probability that a non-chronic offender uses a slur in that game as well.

Take the following chat log:

Code:

---

Player 1: {slur} {slur} {slur}
Player 2: Player 1, please don't say "{slur}"

---

In Method 1, Player 2 would still be flagged for using a slur. Therefore Method 2 seeks to reduce this factor by omitting games entirely in which filtered players used slurs.

This method filtered out 30.19% of the games from the dataset (3257 out of 10790)

https://i.imgur.com/G1YJIts.png

Zooming in on the top 10%:

https://i.imgur.com/UH6hUrI.png

Unrelated Offenses of Chronic Offenders
This section is motivated behind the hypothesis that chronic offenders with respect to slur usage also tend to be problematic players in other aspects of the game.

The metric used to test this hypothesis measured the number of games in which a player used a slur above the average rate.

This metric accomplishes two things:

1. It ignores players that have a high Slur Games / Total Games ratio simply due to the fact that they have played a low number of games. For example a player with 3 games in the data set in which they used a slur in all 3 games would have a 100% slur games / total games ratio - but the actual impact of their slur usage on the community itself is minimal as they only appeared in 3 games.
2. It ignores players that have a high number of slur games only by virtue of having a high number of games. For example a player with 18 slur games would rank in the top 0.5% of players based on number of games in which they used a slur, but if this player has 1000 games within the dataset, they are still well below the average rate of slur usage.

After ordering the players using this metric of number of slur games above the average rate:

• Every player in the top 20 has had a report filed against them
• 18 of the top 20 players have had a report approved against them for unrelated offenses
• 16 of the top 20 players have been watchlisted for unrelated offenses
• 7 of the top 12 players have been banlisted for unrelated offenses
• 4 of the top 12 players have been permabanned for unrelated offenses

Unfortunately these statistics need to be checked manually, and therefore a more comprehensive analysis is not viable at this time. However these preliminary results are strong evidence in favor of the hypothesis that chronic offenders when it comes to slur usage are also chronic offenders when it comes to other offenses.

(Thank you to the staff for helping with these statistics!)

Key Observations

• The overwhelming majority of players do not use slurs at all
  
  • 90.95% of players used zero slurs across all their game
  • Only 3.18% of players used more than 3 slurs across all of their games
  • Only 1.89% of players used a slur in more than 3 of their games
• The racism problem in the game is caused almost entirely by a tiny minority of players
  
  • The top 1.89% of players based on number of slurs used contribute 89.20% of all slurs used
  • When you ignore messages sent by these players then the number of games with zero slurs increases from 63.59% to 87.99% and the number of games with more than 1 slur drops to only 4.68%
  • When you filter out games with slur usage incited by the chronic offenders then the number of games with zero slurs increases further from 87.99% to 91.8% and the number of games with more than 1 slur drops further to only 3.27%
• The chronic offenders when it comes to slur usage also tend to be chronic offenders when it comes to other rule violations
  
  • Every player in the top 20 has had a report filed against them
  • 18 of the top 20 players have had a report approved against them for unrelated offenses
  • 16 or the top 20 players have been watchlisted for unrelated offenses
  • 7 of the top 12 players have been banlisted for unrelated offenses
  • 4 of the top 12 players have been permabanned for unrelated offenses

Raw aggregate data is available upon request.

Post reserved for answering any questions / providing clarifications that might arise.

How are slurs identified?

Code:

---

                for slur in slurs:
                    if slur in line.lower():
                        # marked as as a message with a slur

---

This does nothing to detect obfuscated slurs (although given that using slurs was previously not against the rules, there was little reason for people to obfuscate their slurs).
This also technically double counts messages that have multiple different slurs in the message. Working as intended.

Are player selected names counted?
Yes, player selected names appear as chat messages in the extracted chat logs, therefore they are counted. Although they are not given any additional weight over other regular chat messages.

I like pretty colors and bullet lists. :calix:

This is good work. Like some of your prior work, this doesn't necessarily line up with my experience. I'd be curious to run a similar analysis on my own replays.

How are you identifying slurs?

I'd also note you aren't including player selected names themselves.

Additionally, I'd note that while you identify a small number of games with high slur rates, being in one of those games pretty much ruins it.

I would be curious to look at this point further:
The top 1.89% of players based on number of slurs used contribute 89.20% of all slurs used

In what percentage of games from the data set are at least one of these 2% of players present in?

EDIT

Nevermind this was answered:

This method filtered out 30.19% of the games from the dataset (3257 out of 10790)

So 30% of games had one of these 2% in it.

Finally, I'd be interested in one more thing (promise this is it), for each player for each game the percentage of messages with at least one slur.

It is possible that racists die early (I certainly target them), so I'd want to normalize against a player's total messages in a game.

Quote:

---

Originally Posted by Renegade

This is good work. Like some of your prior work, this doesn't necessarily line up with my experience. I'd be curious to run a similar analysis on my own replays.

How are you identifying slurs?

I'd also note you aren't including player selected names themselves.

Additionally, I'd note that while you identify a small number of games with high slur rates, being in one of those games pretty much ruins it.

---

Good questions, I added them + the answers to the reserved post.

Quote:

---

Originally Posted by Renegade

Finally, I'd be interested in one more thing (promise this is it), for each player for each game the percentage of messages with at least one slur.

It is possible that racists die early (I certainly target them), so I'd want to normalize against a player's total messages in a game.

---

There is no way to tell if a player is alive and spamming racial slurs in game, or dead and spamming them in death chat, so this might be a bit more difficult to test than it might initially seem.

That said, it would certainly be possible to determine statistics based on the percentage of messages a player sends which contain a slur, although I suspect this is more likely to detect people using racial slurs as a vehicle to grief and spam as opposed to people that just use racial slurs often while still playing the game. I'll message you with more details

Can you use whatever you're using to find other information, like night actions and stuff. What is the extent of your system here.

Quote:

---

Originally Posted by ZZorange

Can you use whatever you're using to find other information, like night actions and stuff. What is the extent of your system here.

---

number of games where the beguiler redirected a kill would be an interesting piece of info :D

This system is mostly based on some keywords that would appear during a game I assume.
Don't mark my words though.

MAAAAAAAAN now I want to run an experiment to see the percentage of people saying "any leads" that are mafia.

Quote:

---

Originally Posted by ZZorange

Can you use whatever you're using to find other information, like night actions and stuff. What is the extent of your system here.

---

Next step: how to access the constantly updated replay file, whilst in game, to cheat. :calix:

Quote:

---

Originally Posted by Oberon

MAAAAAAAAN now I want to run an experiment to see the percentage of people saying "any leads" that are mafia.

---

D2 it's 100%
Trust me

Quote:

---

Originally Posted by OzyWho

D2 it's 100%
Trust me

---

:laugh:

Quote:

---

Originally Posted by Oberon

MAAAAAAAAN now I want to run an experiment to see the percentage of people saying "any leads" that are mafia.

---

"Why me?" is pretty much a guaranteed scum too

Quote:

---

Originally Posted by aamirus

"Why me?" is pretty much a guaranteed scum too

---

I've also found that people responding to d1 spam in a negative way tend to be Mafia. I think it's because they're afraid the spammer might be a Veteran and they just react emotionally to it :smile:

Quote:

---

Originally Posted by Oberon

I've also found that people responding to d1 spam in a negative way tend to be Mafia. I think it's because they're afraid the spammer might be a Veteran and they just react emotionally to it :smile:

---

D1 spam often gets you hit by mafia rather than reacted to by mafia

It's the lurkers that are commonly maf (or an evil role of some kind, if a guy hasn't talked for 4 days and hasn't suicided he's evil 99% of the time)

Very cool, nice work!

This is very cool. Did you manually go through each replay and count or did you create a program to parse chat logs for slurs?

This was impressive. 10/10

Would rep if I could : )

This is going to create an interesting discussion where I am studying. Thank you for the information and effort. None of the data surprises me, and it restored some faith in the community.

How did you gather your dataset? For example, are the games spread across different times of day? Were the replays only gathered from active players who were able to send them to you?
Good post!

I like the detailed post. Although I normally do not report racism slurs unless someone spams it constantly where it becomes griefing.

I am not a racist when I say this but, In my opinion, I feel like gamethrowing, rolequitting, and griefing is a way more violent crime to commit in a game rather than saying the N word or a slur.

Mostly kids play this game and they think it’s hilarous to say a slur to rile someone up and start a commotion. I have no issue with people reporting it and getting banned. I think that behavior should be removed from the game. Hopefully, it will calm down soon.

Quote:

---

Originally Posted by Yodaddy2

I like the detailed post. Although I normally do not report racism slurs unless someone spams it constantly where it becomes griefing.

I am not a racist when I say this but, In my opinion, I feel like gamethrowing, rolequitting, and griefing is a way more violent crime to commit in a game rather than saying the N word or a slur.

Mostly kids play this game and they think it’s hilarous to say a slur to rile someone up and start a commotion. I have no issue with people reporting it and getting banned. I think that behavior should be removed from the game. Hopefully, it will calm down soon.

---

Bro.. You really hit it. Being racist online doesn't mean you are racist deep inside. Some people, well like us are just blabbing these slurs to find someone to annoy.. cos you know, it's kinda fun when people are getting mad at someone (k fine I meant me) whom they didn't even know IRL and knowing it was just a fuckin game. So yeah, I slightly disagree with you cos its really fun. :laugh:

Ur right.. But gamethrowing should be carefully tread. I renamed a lot of gamethrowing replays cos it's fun to watch from time to time. But I never submit them cos I know punishing them will greatly affect the player base.

But yeah.. The question is.. Is it worth it though..? Enforcing WL/BL against racism, slurs, etc at the expense of killing the player base? I mean.. Compared to April 2021~ or earlier versus today. The lobby takes ages to fill up now.

Aamirus expresses her opinion on the matter that it's quite the opposite cos there are more lobbies created.. But the thing here is, more lobbies created == more lobbies failed to fill-up == less playerbase.

Let's not kid ourselves.. If we really wanted a "clean" game, we are playing town of salem, not the sc2 arcade.

Also, I assure you that most of the player base here are already adults.. where they returned from the game for the sake of nostalgia. Let's accept it, the RTS era is dead. Most kids are into Minecraft, MOBAs, Battle Royale, FPS, newly released AAA games, and whatever those popular twitch channels are playing.

Paopan out.

How did u gather the data in the first place? Did u use webhook like textractor that extracted all keyword strings in an sc2 replay (?)

Also what search algorithm was used to do this? I can imagine it taking awhile to scan through that many sc2 replay files.

Quote:

---

Originally Posted by Auwt

This system is mostly based on some keywords that would appear during a game I assume.
Don't mark my words though.

---

This is correct.

Quote:

---

Originally Posted by HentaiManOfPeace

This is very cool. Did you manually go through each replay and count or did you create a program to parse chat logs for slurs?

---

No, I did not go through over ten thousand replays and count slurs by hand. I wrote a python script for it :cheesy:

Quote:

---

Originally Posted by Coulson

How did you gather your dataset? For example, are the games spread across different times of day? Were the replays only gathered from active players who were able to send them to you?
Good post!

---

Data set courtesy of aamirus. I did not analyze games by time of day. I am not sure if I am at liberty to describe exactly how these replays were gathered. So direct those questions at @aamirus

Quote:

---

Originally Posted by Yodaddy2

I like the detailed post. Although I normally do not report racism slurs unless someone spams it constantly where it becomes griefing.

I am not a racist when I say this but, In my opinion, I feel like gamethrowing, rolequitting, and griefing is a way more violent crime to commit in a game rather than saying the N word or a slur.

Mostly kids play this game and they think it’s hilarous to say a slur to rile someone up and start a commotion. I have no issue with people reporting it and getting banned. I think that behavior should be removed from the game. Hopefully, it will calm down soon.

---

Ideally people using racial slurs in game don't have to be banned - they get warned (watchlisted) and then they stop and just play like everyone else.

Quote:

---

Originally Posted by WrathCyber

How did u gather the data in the first place? Did u use webhook like textractor that extracted all keyword strings in an sc2 replay (?)

---

Dataset courtesy of aamirus, not sure what I am at liberty to say with respect to how it was gathered.

As for extracting the chats from the set of replays - I wrote a python script to convert all the replays to text files, removed invalid data points (eg duplicate replays, lobbies that didn't have enough players to even count, etc) and then extracted the chat logs from all of those replays.

I used https://github.com/Blizzard/s2protocol for the library to extract info from sc2 replay files.

Quote:

---

Originally Posted by WrathCyber

Also what search algorithm was used to do this? I can imagine it taking awhile to scan through that many sc2 replay files.

---

No special search algorithms needed, the default text search algorithm implemented in most programming languages is plenty fast for these purposes.
It took a very long time (several hours) to convert all the sc2 replay files into text logs - but that is more a bottleneck of the s2protocol than any code I wrote.
Because that conversion took so long though - I made sure to do the analysis in two phases, first convert all the replays to text logs with the meta data I needed for analysis included alongside the chat logs and save those logs as txt files. And then just do an analysis of those .txt files which are orders of magnitude faster to read than sc2replay files.

Quote:

---

Originally Posted by Yuesford

Bro.. You really hit it. Being racist online doesn't mean you are racist deep inside. Some people, well like us are just blabbing these slurs to find someone to annoy.. cos you know, it's kinda fun when people are getting mad at someone (k fine I meant me) whom they didn't even know IRL and knowing it was just a fuckin game. So yeah, I slightly disagree with you cos its really fun. :laugh:

Ur right.. But gamethrowing should be carefully tread. I renamed a lot of gamethrowing replays cos it's fun to watch from time to time. But I never submit them cos I know punishing them will greatly affect the player base.

But yeah.. The question is.. Is it worth it though..? Enforcing WL/BL against racism, slurs, etc at the expense of killing the player base? I mean.. Compared to April 2021~ or earlier versus today. The lobby takes ages to fill up now.

Aamirus expresses her opinion on the matter that it's quite the opposite cos there are more lobbies created.. But the thing here is, more lobbies created == more lobbies failed to fill-up == less playerbase.

Let's not kid ourselves.. If we really wanted a "clean" game, we are playing town of salem, not the sc2 arcade.

Also, I assure you that most of the player base here are already adults.. where they returned from the game for the sake of nostalgia. Let's accept it, the RTS era is dead. Most kids are into Minecraft, MOBAs, Battle Royale, FPS, newly released AAA games, and whatever those popular twitch channels are playing.

Paopan out.

---

Ideally, these players don't get banned, they just get warned (watch listed) at first and then stop spamming slurs. And that works in probably the large majority of cases. Only a minority of that 2% of players that frequently use slurs continue to use them after being warned for it. I believe that removing less than 1% of the playerbase who refuses to stop using racial slurs will hardly have an effect on the game's activity.

On the topic of the game's activity, here is the number of full lobbies per week since the start of this year when the SC2 Arcade came back online:

https://i.imgur.com/7HHW49h.png

Something worth noting is that there has been a very notable drop in activity since April/May. Is this because of rule changes wrt racism? Probably not IMO, but what could be causing it is an interesting discussion nonetheless.

Quote:

---

Originally Posted by LagAttack

Ideally, these players don't get banned, they just get warned (watch listed) at first and then stop spamming slurs. And that works in probably the large majority of cases. Only a minority of that 2% of players that frequently use slurs continue to use them after being warned for it. I believe that removing less than 1% of the playerbase who refuses to stop using racial slurs will hardly have an effect on the game's activity.

On the topic of the game's activity, here is the number of full lobbies per week since the start of this year when the SC2 Arcade came back online:

https://i.imgur.com/7HHW49h.png

Something worth noting is that there has been a very notable drop in activity since April/May. Is this because of rule changes wrt racism? Probably not IMO, but what could be causing it is an interesting discussion nonetheless.

---

Are you including mafia 2.0? The first racism punishments were handed out the first week of march so a real question is what caused the increase in activity in the first place. I think the tracker goes back to end of september, can you include those weeks? I'm curious what the activity was like before blizzard broke the arcade over the winter

Quote:

---

Originally Posted by aamirus

Are you including mafia 2.0? The first racism punishments were handed out the first week of march so a real question is what caused the increase in activity in the first place. I think the tracker goes back to end of september, can you include those weeks? I'm curious what the activity was like before blizzard broke the arcade over the winter

---

https://i.imgur.com/Gml8epi.png

Mafia 2.0 is indeed included.

Quote:

---

Originally Posted by LagAttack

Something worth noting is that there has been a very notable drop in activity since April/May. Is this because of rule changes wrt racism? Probably not IMO, but what could be causing it is an interesting discussion nonetheless.

---

I'm going to assume that your data gathering is accurate cos I'm too lazy about checking it. Nevertheless, you denied it's because of the rule change.. But may I just ask a simple question? What are the things that happened around 5-17 to 5-31 that make the player base left? You have peak lobbies around 5-17 (~440 games per day) compared to the latest data gathered on your bar graph 7-12 (~225 games per day). Any idea? cos I have. This also proves that the majority of players here are adults (normally we should see an influx of players around this season.. Most games showed a positive monthly gain (%) but what happened here is quite the opposite).

Quote:

---

Originally Posted by Yuesford

I'm going to assume that your data gathering is accurate cos I'm too lazy about checking it. Nevertheless, you denied it's because of the rule change.. But may I just ask a simple question? What are the things that happened around 5-17 to 5-31 that make the player base left? You have peak lobbies around 5-17 (~440 games per day) compared to the latest data gathered on your bar graph 7-12 (~225 games per day). Any idea? cos I have. This also proves that the majority of players here are adults (normally we should see an influx of players around this season.. Most games showed a positive monthly gain (%) but what happened here is quite the opposite).

---

May I just ask another simple question?

If you are ready to doubt or explain your opinion on why it went down between 5-17 to 7-12,
I do hope you may have an explanation for the soaring period between 3-1 to 5-17, right ?

The rule change as previously stated took place in early March.
Considering this, wouldn't it be farfetch, and probably to some extent, wrong, to say this rule change is the cause of amount of lobbies' reduction?

Quote:

---

Originally Posted by Yuesford

I'm going to assume that your data gathering is accurate cos I'm too lazy about checking it. Nevertheless, you denied it's because of the rule change.. But may I just ask a simple question? What are the things that happened around 5-17 to 5-31 that make the player base left? You have peak lobbies around 5-17 (~440 games per day) compared to the latest data gathered on your bar graph 7-12 (~225 games per day). Any idea? cos I have. This also proves that the majority of players here are adults (normally we should see an influx of players around this season.. Most games showed a positive monthly gain (%) but what happened here is quite the opposite).

---

The lobby bot was down for a bit on the week of 7-12. So that's why that bar is down so much.

The number of players actually banned from the game due to rule changes is extremely small compared to the size of the playerbase. It is very doubtful that this change had any significant impact.

A better line of questioning would be, what caused the game's popularity to decline from October to November last year? What caused the popularity to increase again in the March to May stretch of time? How do these trends correlate with the size of the SC2 playerbase?

And ultimately there will be natural fluctuations in player count due to a multitude of external forces - independent from any centralized decisions made by the staff or the community

This base data set is all user submitted, right? It isn't like it was regularly sampled over time, or a complete data set. So I would be careful trying to draw any conclusions comparing different time periods.

Quote:

---

Originally Posted by Renegade

This base data set is all user submitted, right? It isn't like it was regularly sampled over time, or a complete data set. So I would be careful trying to draw any conclusions comparing different time periods.

---

LagAttack referred to the lobby tracker bot (for this graph only), so I believe the graph should fully depict what happened, pretty close to a complete data set.
At least partially, since at some seldom times the bot was down and therefore could not track anything for sometime (2 or 3 times iirc).