Facebook, Automated Censorhsip, and Experimentalism
by Michael C. Dorf
My latest Verdict column discusses the recent revelation that Facebook employs a small army of moderators armed with thousands of rules for censoring hate speech. To summarize, I note: (1) FB appears to have adopted the policy partly in response to a public outcry over offensive and outright lethal uses of its platform; (2) in addition, in many of the countries in which it operates, FB is legally obligated to censor hate speech; (3) that obligation means the effective export of other countries' restriction to the US, where our First Amendment protects hate speech as free speech; (4) but because FB is a private company, its censorship is legally permissible here; (5) in choosing to censor via rigid rules, FB follows roughly in the path of US legal doctrine, which, where speech is concerned, generally prefers the vices of rules -- under- and over-inclusiveness relative to their background justifications -- to the vices of standards -- uneven application and risk of abuse by decision makers given substantial discretion; and (6) there is no clearly superior option.
Or, as I conclude in the column: "So yes, Facebook’s rules are ridiculous. But given the legal imperative to censor hate speech in many of the countries in which it operates, Facebook may not have any especially good alternatives."
Here I want to probe a little deeper into the FB censorship regime. I'll speculate on the use of computer code in online censorship. I'll then consider a form of regulatory regime--experimentalism--about which I've written in my academic work.
When I read the NY Times story (linked above) that broke the news about FB, I was amazed that FB uses actual human beings to censor its pages in real time. That didn't seem humanly possible. The Times reports: "Facebook says moderators are given ample time to review posts and don’t have quotas. Moderators say they face pressure to review about a thousand pieces of content per day. They have eight to 10 seconds for each post, longer for videos."
That's crazy. Yet even the moderators' own reports seem to overstate how much time they have per post. Over 2 billion people have FB accounts. In a typical minute, over half a million comments are posted. Meanwhile, the Times story says that FB employs 15,000 moderators. If each moderator works a 40-hour week, that averages out to about two comments per moderator per second. Some comments are no doubt short, but others are long. It is not humanly possible to read and screen hate speech at a rate of two comments per second, nor even at the much slower but still way too fast speed of six or seven per minute.
Perhaps FB moderators ignore comments, focusing only on posts that can be shared. That would allow an enormous amount of material to slip through. And even so, just as applied to the main posts, that still leaves moderators with 8 to 10 seconds per post. Given that many posts include links to longish articles, that's bonkers. FB's system is either extremely leaky or there's an automated screen before material gets to the moderators. And that leads to the next logical question: Why not automate the entire process?
Indeed, the very notion of a system of rigid rules that give moderators as little discretion as possible seems like it's ideal for automation. Instead of using their biweekly meetings to update the documents they send to their moderators, FB's team of engineers and lawyers could be updating their algorithms for screening out hate speech and other verboten posts. And because such a system would not use moderators in real time, FB could divert some resources to the task of automation.
Would that result in a good system? No, it would still be terrible. A picture of Charlie Chaplin in The Great Dictator might get censored on the ground that the computer thought it was Hitler. Neo-nazi or KKK material could slip by simply with a few deliberate misspellings. But then, human moderators working at breakneck speed undoubtedly make similar errors, and the computers, using AI, could presumably learn from their mistakes and get better over time. By contrast, it appears that most of FB's human moderators are short-timers who will not have such a chance for improvement.
To be sure, in order for the FB system to improve over time, it would need two things it apparently lacks. One is some metric for determining and recording when it has made the "right" decisions. Think about machine learning in more artificial environments. AlphaZero mastered chess by playing thousands of simulated games, but of course, there's a huge difference between chess and free speech. Feedback from simulated chess games is determinate. Each game ends in a win, a loss, or a draw. Evaluating whether a particular decision to censor or not censor a Facebook post "succeeded" requires a substantial element of judgment. Thus, in order for a FB automated censorship system to work, particular decisions would need to be evaluated independently through some mechanism that leads to reliable consistent judgments.
Assuming that could be done, there would need to be a second element: some method for communicating results back up to central HQ so the algorithms could be adjusted to be used in future iterations. This tasks seems considerably less formidable than the evaluative task, although at least based on the NY Times story, there is no indication that FB currently has information flowing from the ground-level moderators back up to the lawyers and engineers who make the rules.
Now, as I say in the column, one might well object to the project of censorship on grounds of principle. That's fine. Assume we're talking about some other endeavor that is less objectionable. My point is that the system I've just "designed" is a sort of computer/human hybrid version of an organizational pattern that, in academic work a couple of decades ago, I and my various collaborators (especially Prof. Charles Sabel) called "experimentalism."
The core idea is to avoid the mirroring flaws of rigid rules and flexible standards with temporarily rigid rules that are continually updated based on performance evaluations, so that information flows in both directions. Obviously, computers already existed in the late 1990s, but we described experimentalism mostly as a system for entirely human-mediated systems of regulation (such as agency rulemaking). Today's little excursion suggests a new path for research in experimentalist architecture using continually updated algorithms. Facebook's censorship effort is benighted, but it suggests some intriguing paths forward.
My latest Verdict column discusses the recent revelation that Facebook employs a small army of moderators armed with thousands of rules for censoring hate speech. To summarize, I note: (1) FB appears to have adopted the policy partly in response to a public outcry over offensive and outright lethal uses of its platform; (2) in addition, in many of the countries in which it operates, FB is legally obligated to censor hate speech; (3) that obligation means the effective export of other countries' restriction to the US, where our First Amendment protects hate speech as free speech; (4) but because FB is a private company, its censorship is legally permissible here; (5) in choosing to censor via rigid rules, FB follows roughly in the path of US legal doctrine, which, where speech is concerned, generally prefers the vices of rules -- under- and over-inclusiveness relative to their background justifications -- to the vices of standards -- uneven application and risk of abuse by decision makers given substantial discretion; and (6) there is no clearly superior option.
Or, as I conclude in the column: "So yes, Facebook’s rules are ridiculous. But given the legal imperative to censor hate speech in many of the countries in which it operates, Facebook may not have any especially good alternatives."
Here I want to probe a little deeper into the FB censorship regime. I'll speculate on the use of computer code in online censorship. I'll then consider a form of regulatory regime--experimentalism--about which I've written in my academic work.
When I read the NY Times story (linked above) that broke the news about FB, I was amazed that FB uses actual human beings to censor its pages in real time. That didn't seem humanly possible. The Times reports: "Facebook says moderators are given ample time to review posts and don’t have quotas. Moderators say they face pressure to review about a thousand pieces of content per day. They have eight to 10 seconds for each post, longer for videos."
That's crazy. Yet even the moderators' own reports seem to overstate how much time they have per post. Over 2 billion people have FB accounts. In a typical minute, over half a million comments are posted. Meanwhile, the Times story says that FB employs 15,000 moderators. If each moderator works a 40-hour week, that averages out to about two comments per moderator per second. Some comments are no doubt short, but others are long. It is not humanly possible to read and screen hate speech at a rate of two comments per second, nor even at the much slower but still way too fast speed of six or seven per minute.
Perhaps FB moderators ignore comments, focusing only on posts that can be shared. That would allow an enormous amount of material to slip through. And even so, just as applied to the main posts, that still leaves moderators with 8 to 10 seconds per post. Given that many posts include links to longish articles, that's bonkers. FB's system is either extremely leaky or there's an automated screen before material gets to the moderators. And that leads to the next logical question: Why not automate the entire process?
Indeed, the very notion of a system of rigid rules that give moderators as little discretion as possible seems like it's ideal for automation. Instead of using their biweekly meetings to update the documents they send to their moderators, FB's team of engineers and lawyers could be updating their algorithms for screening out hate speech and other verboten posts. And because such a system would not use moderators in real time, FB could divert some resources to the task of automation.
Would that result in a good system? No, it would still be terrible. A picture of Charlie Chaplin in The Great Dictator might get censored on the ground that the computer thought it was Hitler. Neo-nazi or KKK material could slip by simply with a few deliberate misspellings. But then, human moderators working at breakneck speed undoubtedly make similar errors, and the computers, using AI, could presumably learn from their mistakes and get better over time. By contrast, it appears that most of FB's human moderators are short-timers who will not have such a chance for improvement.
To be sure, in order for the FB system to improve over time, it would need two things it apparently lacks. One is some metric for determining and recording when it has made the "right" decisions. Think about machine learning in more artificial environments. AlphaZero mastered chess by playing thousands of simulated games, but of course, there's a huge difference between chess and free speech. Feedback from simulated chess games is determinate. Each game ends in a win, a loss, or a draw. Evaluating whether a particular decision to censor or not censor a Facebook post "succeeded" requires a substantial element of judgment. Thus, in order for a FB automated censorship system to work, particular decisions would need to be evaluated independently through some mechanism that leads to reliable consistent judgments.
Assuming that could be done, there would need to be a second element: some method for communicating results back up to central HQ so the algorithms could be adjusted to be used in future iterations. This tasks seems considerably less formidable than the evaluative task, although at least based on the NY Times story, there is no indication that FB currently has information flowing from the ground-level moderators back up to the lawyers and engineers who make the rules.
Now, as I say in the column, one might well object to the project of censorship on grounds of principle. That's fine. Assume we're talking about some other endeavor that is less objectionable. My point is that the system I've just "designed" is a sort of computer/human hybrid version of an organizational pattern that, in academic work a couple of decades ago, I and my various collaborators (especially Prof. Charles Sabel) called "experimentalism."
The core idea is to avoid the mirroring flaws of rigid rules and flexible standards with temporarily rigid rules that are continually updated based on performance evaluations, so that information flows in both directions. Obviously, computers already existed in the late 1990s, but we described experimentalism mostly as a system for entirely human-mediated systems of regulation (such as agency rulemaking). Today's little excursion suggests a new path for research in experimentalist architecture using continually updated algorithms. Facebook's censorship effort is benighted, but it suggests some intriguing paths forward.