How can democratic societies host hard conversations about our conflicts in ways that keep important conversations going rather than shut them down? If you’ve been online at all this year, you’ve probably witnessed a heated conversation with deep disagreement, whether it’s about wars, injustice, or even just the question of who gets to speak. 

While some disagreement may be productive, it’s not uncommon to see conversations slip into rule-violating behavior, such as flaming or personal attacks. On Reddit, volunteer moderators are tasked with reducing this kind of behavior. However, they have limited tools at their disposal, relying mostly on punitive measures, such as removing content and banning users. While these tools can help stop violations in their tracks, they may not help users learn how to participate productively in a community in the future. 

  • How can we improve users’ experience of bans and suspensions?
  • What can online communities do to improve diverse dialog on contentious topics?

Few communities know this challenge better than r/politics, one of the world’s largest conversation spaces about American politics. With nearly 8.5 million subscribers, the subreddit is one of the most active and popular communities for political discussions. It has hosted conversation threads for major US events, such as Supreme Court decisions, crisis events, and political deaths, as well as AMAs (Ask Me Anything) with presidential candidates, senators, and entertainers with a variety of political views.

r/politics wants to be a place where political discourse happens; however, because it’s one of the largest spaces to discuss politics in America, it sees a lot of egregious behavior and bad faith contribution. But because the issues at hand are often contentious, the community also gets out-of-bounds contributions from people who were caught up in the moment or didn’t understand the norms. Mods deal with bad faith contributions by issuing bans that permanently prevent people from participating in the future. But to deal with lesser violations, the community uses temporary suspensions to inform people about how serious their non-normative behavior has been, and give them time to cool off. 

However, open questions remain about how effective temporary suspensions are in mitigating future violations and helping people come back to be meaningful contributors to the community. We want to help them with this through field experiments. 

r/politics, with 39 moderators and millions of comments per month, is one of the largest spaces anywhere for people to discuss US political news

Since 2022, we’ve been working with r/politics mods to better understand how to effectively leverage the tools at their disposal to help rule-violating community members participate successfully in their community in the future. As the first step toward designing new research together, the moderators securely shared their mod log which we have analyzed alongside public records from the subreddit between Jan 2020 and the end of December 2021. To protect people’s information, we analyzed the data on secure, access-controlled servers with encrypted data stores, as part of a study that received ethics review from Cornell’s review board.

Why r/politics is so important and conversations can be so fraught

It’s important to understand the scale of r/politics, which is one of the largest single spaces for conversations about US politics in the world— a community with 8.5 million subscribers — more than MSNBC, USA Today have on YouTube. In the U.S., 8% of Americans regularly get news from Reddit

r/politics, with 39 moderators and millions of comments per month, is one of the largest spaces anywhere for people to discuss US political news, especially as news publishers continue to disable comments sections. In 2021, a non-election year in the US, over 200 thousand reddit users posted nearly 1.5 million comments per month on 20 thousand posts. Election years see even higher activity. In the 30 days prior to the 2020 election, r/politics received 3.6 million comments on 49 thousand posts by 374 thousand unique user accounts.

Because r/politics is one of the only places where the average person’s observations about politics could be seen by millions of people, it attracts a fierce competition over opinions and agendas. Consequently, many comments in r/politics get removed for violating community norms —  for example rules encouraging civility, discouraging doxxing, and encouraging recent news only.

During non-election months, the community removed roughly 124 thousand comments per month (8.5%). In the lead-up to an election, that increased to 274 thousand comments (7.6%). It’s possible that the true content removal rates might be even higher, since Reddit’s own systems sometimes remove especially-awful comments before community moderators and community bots notice them.

PostsCommentsComment RemovalsUnique Active Accounts
Non-Election Year 30-Day Average20,0001,450,000124,000208,000
30 Days Before 2020 Election49,0003,595,000274,000374,000
Table 1: r/politics is one of the largest online spaces anywhere for discussing American politics, with hundreds of thousands of people joining shared conversations in the lead-up to important elections. Counts are rounded to the nearest thousand. (Date range: Jan 1 2021- Dec 31 for non-election year, and Oct 4 2020 to Nov 3 2020 for election year)

How does that translate to the experience of commenters? Out of the 51 thousand people who participated in an average month roughly 24% experienced at least one kind of post or comment removal — even if just for submitting a duplicate link. Nearly 4 thousand accounts received a temporary or permanent ban on average, 1.9% of all posters and commenters per month. In the 30 days before an election, twice as many accounts receive a removal, but that’s largely because the community has 80% more commenters, a portion of which would be bots and spammers; according to moderators, removed content from human accounts experience is typically due to civility violations. Since these rates depend on moderator capacity, it’s possible that moderators might not notice all norm violations in these high volume periods.

# Experienced a Comment Removal% Experienced a Comment Removal# Experienced a Post Removal% Experienced a Post RemovalExperienced  Suspension or Ban% Experienced  Suspensionor Ban
Non-Election Year 30-Day Avg50,90024%4,8002.3%3,9001.9%
30 Days Before 2020 Election103,00027%11,5003.1%5,7001.5%
Table 2: During a non-election month, over fifty thousand accounts in r/politics experience at least one comment or post removed by moderators. During the lead-up to a US presidential election, this number doubled, largely because the number of active accounts nearly doubled in this period. The percentage of accounts that received bans was also fairly stable between election and non-election periods. Counts are rounded to the nearest hundredtsc. (Date range: Jan 1 2021- Dec 31 for non-election year, and Oct 4 2020 to Nov 3 2020 for election year)

Who Gets Temporary and Permanent Bans in r/politics and What Happens Next?

In an ideal world, people who do get suspended or banned are able return and participate substantively in a respectful and productive way. To inform future research that tests interventions to support this kind of successful community participation, we decided to analyze differences in behavior for accounts that temporary or permanent bans— and what happens after their accounts are restored.

To study this, we created a dataset that analyzed participation in the six months before and after an account received their first temporary suspension. We also created two other comparison groups: (a) accounts whose first ban was permanent and (b) accounts that were active on the same days as those that received temporary suspensions (Table 3).

what happens after a suspended account is restored? We found that 81% of people in r/politics do come back and participate.

Accounts that receive their first temporary suspension do have key differences from accounts that never received bans in our dataset. Temporarily suspended accounts comment much more and the median % of comments removed is 13 percentage points higher than non-banned accounts—a rate of removed comments that is over 5 times higher. Accounts that receive permanent bans behave in ways that moderators approve even less often, with a median of 40% of their comments being removed.

What happens after a temporarily suspended account is restored? We found that 81% of people do come back and participate after temporary suspensions. Those that return tend to follow the rules at higher rates than all suspended accounts in the period before suspension — though their median rate of comment removal is still 4 times higher than accounts with no bans (Table 3). Furthermore, a quarter of accounts that return do get another ban of some kind (20% of accounts with temporary suspensions), so there’s still huge room for improvement in how the community supports the experience of temporary suspensions.

Total6 months before ban/suspension, if any6 months after unban/suspension
Median commentsMedian % comments removedMedian postsMedian % posts removedMedian days banned% activeMedian CommentsMedian % comments removedMedian postsMedian % posts removed% Suspended again
Accounts with temporary suspensions26,6132517%0100%12.681%138%077%20%
Accounts with no bans
1,970,35193%086%58%22%067%
Accounts with permanent bans26,062840%0100%

Table 3: The vast majority of r/politics commenters do not receive bans. Among those with temporary suspensions, 81% participate in some way after their account is restored, and the median % of comments removed is 9 percentage points lower for restore accounts. Nevertheless, roughly a quarter of all accounts that receive a temporary suspension (20% / 81%) receive a second ban within six months after their account is restored.

Directions for Future Research with r/politics

Based on this research, we’re excited about the potential to develop research with r/politics on ways to improve people’s experience and re-integration to the community after experiencing a ban. We think there is potential for several kinds of studies and experiments, if the community and moderators consent: 

  • Surveys and interviews:
    • Asking people who have experienced suspensions to tell us more about their experience and how to improve it, building on prior research about the topic
  • Norms interventions:
    • Messages that explain what a temporary suspension means
    • Messages with guides on how to contribute productively to the community
    • Messages from different political standpoints that describe the value of a functional shared conversation
    • Messages that help people productively manage their moral outrage, which is one possible driving force behind heated conversations that get out of hand
  • (Your ideas here)

Supporting Other Subreddits to Study Temporary Suspensions with CAT Lab

If you are a moderator of a community that is interested in fielding experiments to test ideas to help people learn to contribute meaningfully to a community after a temporary suspension, please reach out to Sarah Gilbert either through Reddit (/u/SarahAGilbert) or email (sarah.gilbert@cornell.edu). We have rough draft software that could help generate a similar report for your community, as well as support future experimentation this year.

When you write to us, please check the following details. If you’re unsure, we can talk through it with you:

  • Are you making at least dozens of temporary suspensions a month?
  • Do you have at least a year of archived mod logs, for any period covered by the Pushshift dataset? (Reddit’s founding up to April 2023)
  • If your mod log data is more recent, do you have an alternative source of log data that we could work with?