What Can Companies Do For Moderator Well-Being?

What can Trust and Safety teams do to improve the well-being of the moderators who review content, a job that is often outsourced worldwide through third party firms?

Speaking at TrustCon 2022 are a panel of people deeply involved in the systems that organize global labor forces to review and evaluate potentially violating content. Rachel Lutz Guevara is Vice President of Wellness & Resiliency at TaskUs, a company that provides outsourcing for content moderation. Paul Danter (@pauldanter) runs the Digital Content Services practice for Webhelp, a business process outsourcing company that provides Trust and Safety solutions for global companies. Jordan Paul leads Global Outsourcing at Pinterest who is responsible for leading all vendor management and BPO services that support Community Operations. Dali Szostak is head of UX for Google Trust and Safety. Moderating the panel is Charlotte Willner (@helloyouths), Executive Director at the Trust & Safety Professional Association and the Trust & Safety Foundation.

How has the conversation about moderator well-being shifted over the last fifteen years? Charlotte opens by noting how few resources there were when the field was just starting. Rachel observes that the industry has risen into public visibility due to not so good press, and that many companies are now considering preventive mental healthcare in similar ways that first responders are supported.

Charlotte asks whether there are new developments in research on content moderator mental health. Paul remarks that many people have felt like they have been on their own for a long time — it’s also been hard to find research that answer these questions. Paul says that his team is developing research collaborations with academics to study how to support the mental health of moderators working in very different cultural contexts. “The fact that academia is starting to look into these areas is super interesting,” Paul says. 

What can the TSPA do to better support research? Charlotte reflects that when the TSPA was developing the Trust and Safety Library, they also discovered that a lot of the participants in studies exposed to traumatizing material weren’t Trust and Safety professionals- they were students. Rachel mentions that her organization has conducted 22 in-house, not-public studies to improve moderator well-being. She expresses interest in an upcoming Trust and Safety Foundation research coalition to support companies to collaborate on topics such as psychological health.

Charlotte asks Jordan to comment on how platforms can think about moderator wellness when evaluating potential outsourcing partners. Jordan talks about hiring practices- Pinterest tries to hire people who they genuinely believe care about people. Jordan talks about having in-depth conversations with the Pinterest Trust and Safety team when creating criteria for evaluating content moderation suppliers. She hopes that by prioritizing well-being, Pinterest can influence the market for content moderation to value these things. Once a contract is established, it’s also important to hold suppliers accountable, evaluating them so platforms can be confident about how moderators are treated. 

How can Trust and Safety Professionals advocate for each other? Charlotte observes that the TSPA now has more members outside the US than within. Jordan encourages Trust and Safety Professionals to collaborate to advocate for standards of treatment and support for moderators. Dali argues that TSPA should make the well-being of moderators an objective for the companies that provide outsourced content moderation. As an example, she refers to Google’s standard for wellness programs for people doing sensitive content review— something the company tracks in its contracts.

What are the signs of an effective resilience program for content moderators? Rachel notes that companies have typically relied on standard business metrics to evaluate how content moderators are doing. She suggests that wellness programs should be clearly enough defined to know what their goals are, and that companies actually evaluate them. Paul suggests that people sit down with moderators, ask them questions, and find out whether information from corporate trainings is reaching people. He also suggests that Trust and Safety teams make hires from within content moderation teams. 

Charlotte asks the panel for tips and best practices on well-being. Paul mentions support during the interview and training process to identify people’s capabilities early on. He also described the anxiety moderators experienced seeing the number of pending moderation tasks and the positive effect on mental health from artificially reducing that number. Dali talks about the possible benefits of providing people with the minimum information needed to make a decision: blurring, grayscale, and showing people excerpts of larger clips. Rachel also encourages organizations to create more distance from the sensory experience of moderation, create more predictable patterns in people’s experiences, and also supporting wellness at other parts of the process. She also encourages companies to provide training in coping skills.

Charlotte asks for examples of coping strategies that seem good but are actually maladaptive. Rachel mentions that it’s not productive or helpful for people in high-stress jobs who rely on substance use to deal with emotions. She describes things like avoidance as other maladaptive coping strategies. 

People are often drawn to Trust and Safety because they want to make a difference, and who feel like they have to stay even if it’s bad for them. Charlotte asks what mechanisms can help people who are in the field and want to stay but also recognize that it’s becoming difficult. Dali suggests regular surveys, therapy sessions, and other channels for managers to learn about people who are facing difficulties. Dia also thinks it’s important for firms to implement suggestions made by teams. Paul thinks it’s important for team leaders to develop strong relationships with their teams so they can learn about challenges people are facing. WebHelp has started to use bots to seek daily feedback from people about their experiences and offer support when needed. 

How does the challenge of providing content moderators with wellness initiatives vary by culture? Charlotte observes that companies need to hire people in locales that may not have wellness infrastructures or cultures that support people’s mental health. Rachel observes that the Philippines for example doesn’t have enough mental health professionals to support all of the people involved in content moderation work — or even a culture where people are used to receiving that kind of support. 

How can commercial content moderators be supported to talk about issues when they come up? At Pinterest, Jordan says it’s important for the company to understand what issues moderators are facing. Listening well entails checking in regularly with wellness professionals, flying to meet content moderators, and create as many communication channels as possible. Within outsourcing firms (BPOs), it’s important to communicate to moderators that they can communicate to the BPO as well as the client when issues come up— and to create conditions where workers don’t feel like they will face negative consequences for reporting problems. Paul observes that people who have more distance from the daily work of content moderators can struggle to understand the experience—so it’s important to keep that distance short.

What happens when someone is ready to go, either because they really struggled with the content or because they have some other dream? Charlotte observes that there are lasting consequences to the job. Given that, she asks, what kinds of support can be provided? Dali encourages firms to think about these issues across the whole lifecycle of a person’s job, and how can they build an exit strategy into the work? She suggests that people have opportunities to change their role and that they be encouraged to let their employers know about role changes as soon as possible. Rachel points to “opt out policies” within companies where people have the opportunity to change positions while retaining their employment. Factories, for example, wouldn’t expect someone who has a workplace injury to keep doing the thing that injured them or lose their job. According to Rachel, some outsourcing firms have the flexibility to pivot people to a client that doesn’t require content moderation within 24 hours. Jordan also points to post-employment programs that offer support to people over up to 18 months after their employment ends.

Charlotte ends the session by thanking moderators – “You are the real MVPs.”