Study Results: How well do harassment prevention interventions transfer between communities?
August, 2022
Can clear institutional policies against harassment reduce its prevalence in a community? And what side effect (if any) do they have on freedom of expression?
In 2016 as a PhD student, Nathan worked with moderators of the r/science community on Reddit to test this question. In an experiment with 2,190 discussions, he found that the messages increased newcomer rule compliance by over 8 percentage points and increased newcomer participation by 70% on average. In addition to extending psychologists’ understandings of social norms, the study also helped answer whether policies against harassment reduce or increase participation. In this case, visible policies against harassment increased rather than reduced online speech.
Many people have asked us if they could use similar techniques to reduce harassment. As scientists, CAT Lab’s answer has always been “you should test it for your self.” Watching tech platforms and policymakers issue wide-ranging advice based on our results, we have argued that decisions need a better standard of evidence than just one study. That’s one reason behind CAT Lab’s ongoing research with communities — toward our vision of a world where digital power is guided by evidence and accountable to the public.
When communities test interventions for themselves, they can answer practical questions about what works in their context. Those results can then inform a wider scientific conversation about where an intervention works, and where it might not. From 2017-2019, during Nathan’s postdoc at Princeton University, Reddit communities r/futurology, r/ffxiv, and r/science worked with student teams and CAT Lab to repeat the study with r/science.
When communities test interventions for themselves, they can answer practical questions about what works in their context
Why repeat this specific study? Psychological interventions are sometimes effective only for a short period, or only in some contexts. This might be especially true for social norms interventions, since they involve people’s changing understandings of what others expect of them. Since our 2016 study, many things have changed on Reddit, in the world, and within Reddit communities. In 2016, the company banned the pizzagate community. In 2017, Reddit banned several communities for inciting violence, and in 2018, they banned several QAnon groups. In 2019, Reddit updated their content policies to more clearly sanction harassment. The company also increased its centralized content moderation efforts, removing 6% of all content posted to the site in 2020 for violating the platform’s policies across 80,000 communities. After bans and quarantines, Reddit groups that encourage violence found it more difficult to recruit members. And in the spring of 2020, just as we were concluding two of the replication studies, COVID-19 swept through the world.
In this post, written for our community collaborators, we report on what we learned about influencing newcomer behavior by posting the rules, across three new field experiments on Reddit. We are excited to hear feedback from communities and incorporate what we learn into research publications we plan to submit for publication later this year.
How these studies worked
For this study, we used CAT Lab’s CivilServant software, which observes conversations in an online community in close to real-time. As new conversations were started, the software “pinned” messages to the top of the discussion on desktop and mobile that reminded people of the rules (see the experiment plan here). We then recorded three outcomes that mattered to communities: the number of newcomers per post, whether a given newcomer comment was removed or not, and the total number of newcomer comments removed per post. In r/science, we omitted large-scale Q&A posts called “AMAs.”
All of the messages we tested included several basic components:
- Welcoming newcomers
- State community rules
- Explain the consequences of rule violation
- Offer credible evidence that rules will be enforced
| r/science 2016 | r/science 2020 |
| Welcome to r/science! Our team of 1,500+ moderators will remove comments if they are jokes, memes, off-topic, or medical advice (rules). We encourage respectful discussion. | Welcome to r/science! Our team of 1,500+ moderators will remove comments if they are jokes, anecdotes, memes, off-topic or medical advice (rules). We encourage respectful discussion about the science of the post. |
| r/ffxiv 2019 | r/futurology 2020 |
| Welcome to /r/ffxiv! Gyorin the Gunbreaker asks you to keep in mind Reddiquette, follow our community rules, and be civil with your fellow Warriors of Light. Threads on bad experiences with other players (even anonymous) as well as hate-based comments such as personal attacks, bigotry, hate speech, and name shaming are subject to removal by the moderator team under rule 1. Please report any rule violations; the moderator team will review them as soon as possible. |
Welcome to /r/Futurology! To maintain a healthy, vibrant community, comments will be removed if they are disrespectful, off-topic, or spread misinformation (rules). While thousands of people comment daily and follow the rules, mods do remove a few hundred comments per day. Replies to this announcement are auto-removed. |
The new studies also had one major difference from 2016. Unlike the original study, Reddit changed their platform so we couldn’t mask moderators to which posts had received the intervention. Previously, we could be confident that the norms messages weren’t influencing moderators to review comments differently in discussions that posted the rules. That was no longer true in these replications, creating a potential source of bias.
What we learned
In both experiments in r/science, posting the rules increased the chance that first-time commenters would follow the rules. In 2016, we found a 7 percentage point increase; in 2020 during the early stages of the pandemic, we found a 4 percent increase. We did not find an effect within the Final Fantasy XIV community, where newcomer comments were already rare and almost never removed by moderators— leaving little room for improvement. In r/futurology, where removals are more common, we also did not observe an effect.
In addition to looking at the effect on individuals, we also looked at the overall effect on posts, counting the number of newcomers and the number of newcomer comments removed.
When we first imagined this study in 2016, communities and policy scholars were worried that posting rules against harassment might reduce participation. In the first study, we found that visible rules increased the number of first-time commenters. In the two follow-up studies, we didn’t observe a difference in participation rates when posting the rules. In no experiment did posting the rules clearly reduce the number of newcomers.
We also investigated possible effects on the workload of moderators, an outcome that depends on an intervention’s influence on participation counts. In the first study with r/science, posting the rules increased the number of newcomer comments removed because the number of newcomer comments increased overall—even if individual newcomers were less likely to violate community rules. In the 2020 r/science study, the newcomer rate was not measurably different, so the number of removed newcomer comments declined alongside the probability of individuals violating the rules. In in r/ffxiv and r/futurology, we did not observe an effect on the rate of newcomers or the rate of newcomer comments removed.
Assessing the role of COVID-19 on these studies
Since the r/science and r/futurology studies in 2020 occurred during international lockdowns amidst early spread of COVID-19, we wondered if the pandemic might have influenced the results in some way. Fortunately, we have data to help answer the question. When developing studies with communities, CAT Lab collects initial data to help plan the study. We can use that information to compare community participation during the study (in the control group) to the planning period.
What was different? During the first weeks of the pandemic, r/science experienced a steady reduction in the number of newcomers per day (minus 1.1% per day, p<0.0001), resulting in a newcomer rate that was substantially smaller than the subreddit regularly experienced just a few months earlier. Futurology also experienced differences, receiving roughly half the number of posts per day in the spring of 2020 compared to one year earlier.
| Purpose | Start Month | Subscribers | Mods | Posts per day | Comments per post | Newcomers per post | Newcomer comments removed per post |
| r/science 2016 study | |||||||
| Control Group | 08/2016 | 12.2 million | 1,196 | 76 | 22 | 6 | 4 |
| r/science 2020 study | |||||||
| PlanningData | 10/2019 | 22.5 million | 1,546 | 58 | 117 | 36 | 16 |
| Control Group | 02/2020 | 23.2 million | 1,550 | 55 | 74 | 9 | 3 |
| r/futurology 2020 study | |||||||
| Planning Data | 12/2018 | 13.4 million | 27 | 90 | 38 | 7 | 1 |
| Control Group | 01/2020 | 15 million | 34 | 47 | 53 | 10 | 2 |
| r/ffxiv 2019 study | |||||||
| Planning Data | 01/2019 | 209,416 | 10 | 166 | 29 | 1 | 0 |
| Control Group | 07/2019 | 262,556 | 18 | 246 | 30 | 1 | 0 |
We also looked into splitting the data into time periods before or after the lockdown. We didn’t find compelling reasons in the data to do any fancy lockdown-related analysis. We did remove 2 randomization blocks from r/science that contained 2 discussions with huge comment counts in the days before the WHO declared COVID a pandemic. Since our software assigns randomizations over blocks of time, we could remove these blocks while preserving the overall integrity of the randomization scheme.
Statistical notes
This report uses a simpler “difference in means” method to estimate the effect on individual newcomers in this study, compared to the method used in the original experiment. Since the 2016 study, statisticians have updated advice for estimating binary outcomes. The previous analysis approach offered a bit more precision, but it’s harder for the general public to interpret. We also increased our sample sizes in subsequent studies to account for the change. In this post, we also apply this difference in means method retroactively to the 2017 study.
Why might the results be different between subreddits?
If the rule-posting intervention continues to be so effective within r/science, why wouldn’t we see similar results elsewhere? Here are a few possible explanations:
First, it’s possible that moderators in the follow-up studies were also influenced by the intervention, since we could not prevent them from seeing the welcome message. If the message reminded moderators of the rules, they might have enforced them more strictly for the treatment group than the control group, causing us to under-estimate the actual effect. To investigate this possibility, we set up a preliminary analysis of moderator behavior before, during, and after the 2020 r/science study to disentangle this from the possible effects of the March 2020 COVID lockdown or other factors.
In r/ffxiv, newcomer comments were so rare and moderators were removing so few comments that it would be hard to observe meaningful differences. That was true when we initially imagined the study with r/ffxiv moderators. In the behavioral sciences, it’s still valuable to test whether an intervention can be effective at changing rare occurrences, and the r/ffxiv community was growing substantially. In this community, posting the rules doesn’t appear to have done any harm.
In r/futurology, we don’t find an effect on newcomer behavior. However, due to a moderation log data collection glitch during the early days of the pandemic, we had to remove roughly 300 randomization blocks from the tail end of the study, amounting to 3,600 posts. With only 74% of the posts we needed for a statistically-significant result, this result is not surprising. We are exploring ways to fill in the moderation log gaps before submitting an academic publication. Despite this limitation, the findings in r/futurology are consistent with what we saw in the r/science studies, with a positive estimate for the effect on newcomer behavior—one that we can’t differentiate from no effect at all.
Across communities, it is also likely that some of the newcomers in these studies were “alt” accounts— new accounts registered by banned users. While Reddit tracks these accounts, they only provided communities with a tool for detecting ban evasion in August 2022. Since an unknown percentage of accounts in the study are not newcomers but actually people who are deliberately evading bans, we expect the study to under-estimate the effect on rule compliance.
Finally, we should also note that each community has different norms. While all of these communities have norms against harassment that Reddit’s own policies have aligned with since 2016, r/science and r/futurology have more unique norms that people might not have learned elsewhere: r/science has rules against jokes, memes, anecdotes, and medical advice; and r/futurology has rules against misinformation. The r/science and r/futurology messages also include stronger evidence that they enforce those rules. It’s possible that differences in results might arise from differences in the rules and how they are communicated.
The future of studying social norms online
Across these four studies in reddit communities, we find that making community norms visible does influence the first actions of newcomers to communities on average. At the same time, we have learned some of the circumstances where these interventions may not be as necessary or helpful — especially where norm violations are rare. By working alongside communities (including several related upcoming studies), we have also learned valuable lessons for future research that could help reduce online harassment while advancing scientific understanding of social norms.
First, we think it’s important for future research to learn people’s beliefs directly— both people’s personal beliefs about acceptable behavior and their beliefs about what others consider acceptable. To do this, we have started to incorporate surveys into our more recent studies. Supported by the Templeton World Charity Foundation, we are working to archive, report, and improve survey response rates for online behavioral research in hopes of developing best practices for linking surveys to behavioral data in research about online conflict and forgiveness.
Second, because online platforms collapse contexts, digital environments also collapse norms— creating arenas for conflict between different ideas about how to behave—amplified by algorithms. We’ve seen evidence for these conflicts in conversations with communities and in some of our early work to link surveys and behavior. CAT Lab’s Lucas Wright and William Frey are working with reddit communities to study what happens when norms collide in algorithms and online behavior.
Reddit has continued for years to withhold machine-readable viewership and subscriber data from online communities
Finally, online social norms research has a measurement limitation, created by the choices of tech platforms to grant people access to their own data. While reading this post, you might have noticed that our studies don’t include information about people who received the intervention but chose not to comment. That’s because Reddit has continued for years to withhold post-level, machine-readable viewership and subscriber data from online communities (although it’s standard on other platforms). Due to Reddit’s limited support for communities to understand themselves, we are limited in our ability to help communities answer questions—even when that community labor helps the company’s bottom line.
Learning more
To learn more about this study, you can view the analysis code and follow the research page for our replication studies, which includes the latest links to community conversations, research plans, statistical code, and scientific reports.
If you’re interested to collaborate with CAT Lab in your community, please reach out to us at research@citizensandtech.org.