How can Trust and Safety teams learn from public health when imagining and deciding on interventions? And how can that translate into procedures that teams can practically follow?

We’re here at TrustCon2022, where the the opening keynote speaker is Del Harvey (@delbius), former Vice President of Trust and Safety at Twitter. Del has devoted the better part of the last 14 years to Trust & Safety work at Twitter, where she founded, built, and led the company’s global T&S team. These days, Del does consulting on content moderation, online safety, freedom of expression, privacy, and what it takes to lead and scale a global team.

Del opens by reflecting on how much Trust and Safety has matured in the last fifteen years. Many of the early people in those roles needed to learn many things the hard way. For example, says Del, don’t make the country field in your law enforcement forms a freeform text, or you will spend an inordinate amount of time sanitizing your transparency report. People do this work, says Del, across the private sector, government, and civil society. 

Del says she’s optimistic about the future of Trust and Safety because of how much people care about the work and because of the community’s willingness to share with each other what they’ve learned.

How do other similar fields handle high stakes, fast-moving issues with murky information? Del found companions in public health and epidemiology. She ultimately came to the conclusion that at its core, Trust and Safety’s purpose is to serve health – the people who use platforms, the people who work at companies, and the wider world. This means more than just addressing bad things. It’s not just the absence of disease, disorder, or injury. In its ideal state, health is complete mental, physical, and social well-being. 

Five Responsibilities for Trust and Safety

Del identifies five distinct responsibilities. The first is to prevent risk factors, targeting underlying systemic conditions to prevent those risks from ever developing. The second is to reduce risk, eliminating or reducing existing risk factors, enhancing protective factors to build people’s resilience to risk. Trust and Safety should also detect harm and intervene to start them, as well as mitigate harms. Finally, Trust and Safety should do these things in the “right way” – considering ethics and ensuring that interventions don’t cause more harm than good.

What does it mean to prevent risk factors? Del compares Trust and safety to preventive healthcare. In public health, experts might suggest increasing neighborhood walkability to improve people’s health. In online environments, this might involve thinking about broader changes to people’s user experience, such as making it easy for people to confirm their email addresses so they don’t lose access to their account.

What does it mean to reduce risk factors? In public health, says Del, a vaccine or seatbelt laws might reduce the risk of serious infection or serious injury. An account platform that is experiencing a large number of account compromises might send people an email asking people to confirm their account as a preventive intervention.

How might platforms detect harm and intervene? Del argues that platforms should detect harm and halt its progression at the earliest possible point. In public health, this involves screening and identifying disease early so people can get treatment before it can spread. In Trust and Safety, teams can monitor dumps of compromised passwords and notify people if their account has been compromised. 

Another step is harm mitigation, says Del. In healthcare, harm mitigation interventions happen once a disease presents symptoms. In Trust and Safety, teams could require people to use two factor authentication after an account has been compromised to limit the damage.

How can Trust and Safety professionals choose which approaches to take across all of these areas? In public health, a combination of ethical principles and research provides guidance. For example, the medical profession needs to limit the excessive use of antibiotics lest they encourage antibiotic-resistant diseases. Del thinks Trust and Safety professionals need to balance all of their tools with a combination of research and ethical principles.

Del next compares trust and Safety to the health bar of videogames, saying that Trust and Safety teams should help people to keep their health bar longer and expand their capacity to experience risks. T&S teams can work to protect health through preventive measures that remove negative health influences and reduce the likelihood of someone encountering a given risk.  Harm prevention interventions focus on specific groups to reduce the chances of someone being affected by a specific harm, says Del. If someone is already affected, they interrupt or slow further damage— or even repair harm. Finally, health promotion interventions empower people to take charge of their own health, says Del. These initiatives build people’s capacity to better handle the risks they encounter. Del thinks this is an area of significant opportunity for trust and safety.

By focusing on health protection, harm prevention, and health promotion, Del says, Trust and Safety Teams can move beyond the Whack-a-Mole that teams often find themselves handling. How can teams actually do this? Del summarizes a template she created for people to complete when proposing new Trust and Safety interventions. This template invites proposers to describe the purpose of the initiative, the evidence supporting the initiative, the rationale, evidence collection, the ethics, and its expected impact on equity issues. An implementation section of this template also requires proposers to document the criteria for using the intervention, legal issues, the changes expected, and other details. 

Del hopes that this template might help spark a conversation about what questions to consider when imagining an intervention— as well as a guide to organizational decision-making. Del has published the templates to her website at delbius.com