Today, FastCompany announced that CivilServant was a finalist in their Innovation by Design awards. Social science projects aren’t usually considered for design awards, so I’m glad that they took CivilServant seriously as design innovation. The idea of design is a central concept in my MIT Media Lab PhD dissertation on governing human and machine behavior in an experimenting society. I still remember a committee member writing in the margins of an early thesis draft, “what do you mean by design?”

As you may know, CivilServant scales society’s capacity to discover effective ideas for a fairer, safer, more understanding internet. Over the last few years, I have done this by supporting citizen behavioral scientists to test practical ideas for responding to human/algorithmic misinformation, preventing harassment, reducing discrimination, and auditing algorithms, collaborating in some cases with communities of tens of millions of people. CivilServant also offers independent, public-interest testing on the social impact of internet technologies. FastCoDesign covered the project back in March. CivilServant is now being incubated by the borderless citizen media organization Global Voices.

How is any of this design? In this post, I try to answer that question.

(PS: CivilServant is currently hiring a lead data architect)

Design as the power to structure society

Technologies structure our social worlds and are shaped in turn by the people who live and work with those structures. Online, our behavior is structured very closely, and the paths we take are monitored and shaped by platforms and their algorithms. Creating those structures is one of the great design endeavors of our time.

Because social technology designers are able to reach into the daily lives of billions of people, we need ways to use that power wisely. We also need to ask how to manage that power in democratic societies.

The social sciences are essential to design because our best intentions are often mistaken and our best ideas often turn out differently than we expected. Smart ideas for beneficial behavioral nudges and AI systems can cause the opposite outcomes and reinforce problems they set out to solve. Like other work in the emerging area of behavioral design, CivilServant contributes to design by developing empirical evidence on the outcomes of social technologies, asking questions defined by the communities affected.

Behavioral citizen science

Social research itself is also something we design, and that’s often harder to see. The mathematics of statistics or the conditions required for a valid experiment can seem like standards to meet rather than processes with politics that are themselves worth experimenting on.

When behavioral researchers view the way we work as something that is designed according to a set of values, goals, and constraints, we can imagine new ways to work. To create CivilServant, I studied the history of social experiments to find moments when founding researchers questioned the role of their methods in democracy. To develop my own project, I found inspiration in leaders like Kurt Lewin, Karl Popper, Donald Campbell, and Ann Oakley, who imagined democratic directions for social experimentation.

…as communities ask new questions, citizen behavioral scientists grow collective knowledge about the outcomes of ideas for a fairer, safer, more understanding internet.

Why redesign experiments? As the behavioral sciences have become central to the work of running large online platforms, computer scientists have developed experimentation infrastructures that can do tens of thousands of studies a year. These systems and the engineers who design them have created a genuine leap in the the ability to learn how to influence human behavior. To achieve this scale, they have built on a century-old idea of control through communication that comes to us through the industrial revolution. Sometimes companies use this power beneficially. Other times, companies try to leverage their design power to shape our behavior against our own interests.

While the ability to do social research can help us make wiser design choices, it also tends to concentrate power. When researchers try to maximize the impact of our genuinely-valuable work, we have a tendency to want more power than a reasonable democracy can be expected to give us. In the 1970s, Carol Weiss documented researchers’ mistaken beliefs about how democracies makes changes based on research. When communities and governments don’t implement their suggestions, researchers can become frustrated and demoralized.

Many researchers mistakenly believe that research helps solve policy problems, according to Weiss (1977)

Today, while findings from large-scale online experiments aren’t always applied automatically, this direct route to impact is more common, with little public deliberation and negotiation in-between. In most cases, the public isn’t even informed about the research or the ways the research defines designs that direct our behavior.

With CivilServant, I am redesigning large-scale, online behavioral research in ways that are led by internet users, independently platforms. CivilServant is modeled after large-scale corporate A/B testing systems, and it’s also inspired by citizen scientists who do conservation and environmental research. Behavioral design with CivilServant works through the leadership and deliberation from the communities most directly affected. Here’s what the design process looks like for CivilServant, something I call the community research spiral.

In the community research spiral, people affected by behavioral design define the research and deliberate decisions based on the results.

New studies with CivilServant start when an online community, such as the science community that worked with us last year, asks us to help them test the outcomes of an idea for improving behavior online. If the CivilServant software can already connect to the platform that hosts the community, moderators share access by inviting our bot into their community to collect data with their consent. We then talk about the study design; where possible, we host public consultations with the whole community. After an ethics review, we start the study, which is usually coordinated by our bot.

During the study, the moderators may end the study at any time. Once the study concludes, we analyze the data, share the data with a community, and support their deliberations about whether anything should change in light of what we discovered. Over time, as communities ask new questions, these citizen behavioral scientists grow collective knowledge about the outcomes of ideas for a fairer, safer, more understanding internet.

Millions of people have already benefitted from our community-led research. Across reddit, roughly a hundred communities signed up for a reddit-led trial of moderation practices to replicate our findings with the science discussion community. Several other platforms have also rolled out new features inspired by our work. Our process still has rough edges, and I’m sure we’ll make mistakes as we grow, which is why I’m grateful for the involvement and feedback from people in the communities we work with.

Overall, CivilServant contributes to design in two ways: First, with each new study, communities create new knowledge on the outcomes of ideas for beneficially shaping human and AI behavior. Second, beyond individual studies, we are redesigning processes and technologies of behavioral design to be led by the public and accountable the people affected.

I’m grateful to the FastCompany Innovation by Design Awards for recognizing what we have achieved so far, to Merry Mou, who co-created the CivilServant software with me, and to everyone who helped us get here.