Last week, Meta announced that they had removed over 60,000 Instagram accounts for engaging in sexual extortion, including of minors, on the platform. While the company made the final call, this outcome was preceded by research from independent, open-source investigators and journalists. Within Meta, the company said the decision was the outcome of new technical inputs their engineers have developed to detect these scams and lengthy investigations from their experts. The removal of a complex network of scammers took deep investigations and collaborations across teams, inside and outside of the company.

I bring this up because oftentimes, it seems to me that people outside of this industry think of content moderation purely as content review — policy teams set rules and then humans or AI review pieces of content one by one looking for violations and removing them. And this is certainly part of it. But decisions about acceptable online behavior and procedures for keeping people safe online are becoming increasingly complex, involving more and more skillsets and forms of expertise. As this field expands and evolves, I’m reflecting on what forms of expertise are being elevated and where the boundaries of a newly professionalized line of work are being drawn.

Nowhere is this more apparent than TrustCon, a large, global conference for people who work in trust and safety that took place in San Francisco last week, hosted by the Trust and Safety Professional Association (TSPA). The TSPA is a non-profit that exists to support the range of people who work in content and behavioral policy and enforcement at platforms and to create a shared community of practice that those workers can draw on and contribute to. The work of trust and safety is very closely linked to the research we do at CAT Lab, since our goal is to produce community science that helps make a safer internet, and trust and safety professionals work everyday to protect users on their platforms from a variety of harms. 

Some of this growth is likely due to growth within the field: hiring in new trust and safety roles and the establishment of new trust and safety teams. But much of the growth is also, I would guess, due to growth of the field. As TrustCon becomes more and more established, sectors and professions that previously may have thought of the field narrowly as content review and moderation are seeing their work as trust and safety — lawyers advising clients on audits required by the EU’s Digital Services Act (DSA), background check companies hoping to capitalize on identity verification and age assurance laws, machine learning engineers who work on bias in decision-making, open-source intelligence investigators, among many others.

This is a sign, I would argue, that the process of field-formation in trust and safety is still ongoing. It’s not yet clear where the boundaries are, which makes it an important moment to ask: Who is a trust and safety professional? And whose expertise is valued by this field?

Ten years ago, the phrase trust and safety existed in the industry, but it wasn’t universal in the way it seems it is becoming now. Teams that were doing this work were sometimes called things like “community support” or even “anti-evil operations.” Sometimes they were simply folded into the legal department. Today, we all recognize the work those teams did as clearly trust and safety, but if you had asked them at the time what field they work in, I doubt you would have gotten a consistent answer. Without a shared field, whatever information sharing on best practices and resilience, on mutual aide and support was ad hoc and informal. 

This is where the field forming function of TSPA and TrustCon makes an incredible impact. It’s impossible to be at TrustCon and not feel the sense of community, shared understanding, and common purpose it makes possible. As many speakers said throughout the three days, resilience is key to doing this incredibly hard work, and I can’t help but think this conference and the community it fosters is contributing greatly to that resilience. 

As a field crystallizes, however, certain markers of status and expertise become more important, and boundaries start to form that are more firmly established than in a loose collective. As the field grows, it strikes me that many of the strongest forces acting on the field (the DSA and other new regulation around the world, AI enthusiasm boosted by large language models, new waves of VC investment in start-ups in this space, etc) are pushing the field toward more established, more recognizable forms of expertise from adjacent fields like law and security. This is the kind of expertise one acquires from an advanced degree or a previous career in government, public relations, or consulting. 

I’m not arguing that this is a bad thing — these forms of expertise are inevitably going to be a part of a profession that is increasingly regulated and automated. And incorporating existing knowledge from more established fields will help trust and safety mature more quickly. The news of Meta’s removal of tens of thousands of scammer accounts that I opened with demonstrates the value of this mix of expertise. 

But for a field born out of the expertise cultivated by content policy and content review workers in that time before there was a universal name for what they were doing, it would be a shame for this process of professionalization to devalue the knowledge of the people doing that frontline work today. The same goes for the knowledge of volunteer moderators of spaces like subreddits and Facebook groups — ordinary people who have slowly developed real expertise about how to create safe experiences online over countless hours of unpaid work, who will likely never be recognized as professionals by platforms for the work they do. 

Next year, when TrustCon surpasses 1500 attendees and the year after when it cracks 2000, who will be in attendance for the first time? What will be the source of that growth? What forms of knowledge will the field gravitate toward, and how will this drive the priorities of a profession that plays an essential role in keeping us all safe online? The future of online safety — and even of what counts as safety — depends on this.

I’m continuing to explore these questions in my dissertation research. If you work in (or near) trust and safety and have thoughts on any of this, please reach out to me.