This week, I’m attending TrustCon: a global conference for trust and safety professionals. I’ll be liveblogging some of the panels I attend.

The rapid proliferation of generative AI and LLMs — and concerns about their safety — has brought work on content policy and work on responsible AI into contact with one another. But these bodies of work were developed largely separately in fields of trust and safety and computer science, respectively. How can these two communities overcome this siloing and better communicate best practices and findings?

This panel brings together experts from both fields to discuss how they can best work together. As a researcher who works on both content moderation and AI policy, I’m especially interested to hear what they have to say and compare it to the conversations I heard at the ACM’s Conference on Fairness, Accountability,and Transparency last month. 

Vaishnavi J is the founder of Vyanams Strategies (VYS), helping companies, civil society, and governments build healthier online communities for young people and was formerly the head of youth policy at Meta. Dave Willner is currently a Non-Resident Fellow in the Program on Governance of Emerging Technologies at the Stanford Cyber Policy Center and, in the past, led trust and safety teams at Facebook, Airbnb, and OpenAI. Alex Rosenblatt, the moderator of the panel, is a founder of SafetyKit, a startup that automates complex Trust and Safety reviews and was previously a trust and safety product manager at Stripe, Meta, and Airbnb. Henriette Cramer is the co-founder of PaperMoon.AI, an AI quality and safety startup based in San Francisco, and was formerly the Director of Algorithmic Impact and Responsibility at Spotify.

Opening remarks

Henriette opened the conversation by observing that over the last few years there have been a huge number of professional communities entering the AI safety space, and this creates a real translation problem. She says that she goes to AI conferences where people have never heard of trust and safety. But she also goes into trust and safety spaces that are reinventing work on transparency that AI researchers have been thinking about since the 1960s. 

Dave shared that we are starting to see this bridging. He says that a lot of the safety work in AI has traditionally been more academic because the systems weren’t widespread. These researchers think about things in a theoretically abstract way, but trust and safety professionals think in very practical terms and are forced to grapple with what happens when millions of people use your systems. For example, he references what Dr. Rumman Chowdhury said in her keynote address earlier today — that reinforcement learning with humans is fundamentally a content moderation problem, and this doesn’t occur to people who have never encountered moderation. 

Vaishnavi points out that AI companies are often thinking about the worst of the worst harms for AI — like CSAM and exploitation — but not the journey of using their product. They are less used to thinking about issues like overall wellbeing that are a result of how a child might use their product. She argues that this is replicating a mistake social media made in the early days. In her work on thinking about how to make chatbots safe for kids, she draws a lot from work on content moderation. For example, in thinking about privacy questions like what information to share with parents when a child interacts with an AI chatbot. She says we can take a lot of those lessons from TS and apply it to this new work on AI safety. 

How to make general principles useful for specific interventions

Dave argues that risk is a function of the design of a product itself, not the technology it’s built on. A lot of people think AI itself is scary, but that’s only as true insofar as the design of the product you are deploying is risky. For example, he says that having a person interacting with a chatbot is a lot safer than having a person interacting with everyone on the internet. The harm in that case is not about AI, it’s about interactions that can become harmful. 

Henriette says that when she was at Spotify, they had a large-scale algorithmic impact assessment effort, a common intervention in responsible AI literature and government recommendations that CAT Lab has written about before. After 100 of these assessments, she says you realize that people are often excited to help T&S teams and make their products safe. But they may have very different expectations about what the goal looks like. They might be enthusiastic, but they might have a lot of misconceptions about the complexity of making their products safe. Vaishnavi agreed with this, saying that the more specific the recommendation the more valuable it is. She says that what gets lost in translation is when we take broad principles that everyone can agree on into actionable steps. 

Dave then pivoted to an observation that oftentimes, the AI industry wants to make a model that can do everything for everyone at all times. But he says safety requires particular interventions to a person’s circumstances. AI researchers talk about alignment with human values, but we have to ask what are human values? How do you operationalize that? It isn’t obvious to people outside T&S that operationalizing values is very hard, and the tradeoffs aren’t obvious.

Alex asked whether that challenge comes from a lack of shared language, and Dave said yes, in part because we are only just beginning to figure out language to describe what T&S does as a community. He argues that we have to do a better job of explaining externally about what our project is and what bad content is. Vaishnavi echoed this, saying that we’re moving into a realm where we have a community like this to explain it. She says that people used to think that what T&S teams do is really easy, and we’re getting to a point where we can better explain complexities. 

Henriette observed that it would be helpful to have more conversations about what the explicit expectations are for different fields. She described a situation where two teams think the other is ultimately responsible for implementation: engineering teams might think that we can send all of the data to T&S and they will handle it. T&S teams might think that they are just the policy people who set the rules and product teams develop the implementation. She says we need to be clear about these expectations. 

Dave agrees, saying that with better communication, deploying an AI product safely needs to be built throughout the entire stack — from what the model is trained to refuse, to what mitigation is built on top to identify failures, to what enforcement looks like. To do this, he argues we have to show up with a willingness to learn how new technologies work because AI is very different from the standard engineering cycles many of us are used to. 

Vaishnavi has found this to be true in her own work. She says that when her team goes into new companies to help them think through these challenges, they are most effective when they get very specific about mitigations. They walk them through the user journey through their product, getting very specific about places in the user journey where they might experience harms. 

Closing thoughts

Dave ends on an optimistic note, arguing that TS as a field should be excited about generative AI. He says that generative AI is really good at content analysis and T&S is about content analysis. He thinks we should engage with this tech, not shun it. Vaishnavi is excited for AI to move forward conversations about safe design, and hopes these concerns will cascade to other areas, like social media and marketplaces. Henriette points out that even on this panel, they’ve been using terminology differently. She says it’s exciting that we’re starting to point out these differences and think about how to bridge them.