Toward Greater Content Moderation Transparency Reporting

Freedom of expression is enshrined in almost every constitution globally. It is explicitly reflected in the U.N. Universal Declaration of Human Rights and is protected by the International Covenant on Civil and Political Rights. According to the Universal Declaration, “everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers.” As expression increasingly takes place in the digital domain, the struggle to balance the freedom of expression with the need to moderate harmful content online has become increasingly challenging. Beyond questions of the substance of content moderation, society faces vexing questions about which part of the online information ecosystem is best positioned, if at all, to manage and moderate malicious content. 

Today, society relies largely on social platforms, like Twitter and Facebook, to do the job. But that reliance is neither inevitable nor inexorable. Since 2020, the American University Washington College of Law’s Tech, Law & Security Program has been assessing the question of whether or not content moderation might also be conducted elsewhere in the multilayered system that makes up the internet, and, if so, to what effect.

A major challenge in making that assessment is the critical lack of information on how content moderation can be carried out online through nonplatform parts of the wider ecosystem. This lack of information is particularly stark when compared to the platforms, which have responded to significant public and legislative scrutiny with a relatively high level of transparency (however imperfect that transparency may be). Facebook, for example, established a quasi-adjudicative process for reviewing the decisions it makes. Transparency, in this context, is a positive social good. It both describes and enables evaluation and accountability for content moderation decisions. It also provides valuable information on content moderation for policymakers, academics, and users. 

Other parts of the online information ecosystem are not nearly so transparent about their activities. On the one hand, other providers, outside of social platforms, are manifestly making content moderation decisions. For example, after 8chan—well known for its lack of standards toward harmful online content—was tied to a hate-motivated mass shooting, Cloudflare, an internet backbone service provider, ceased providing network services to the platform. Likewise, in the wake of the Jan. 6 Capitol insurrection, Amazon Web Services (AWS) terminated its cloud hosting agreement with Parler, the alt-right social media platform. In both of these cases, backbone internet service companies demonstrated their capacity to moderate content online by making parts of the online information ecosystem that they deemed harmful effectively (if temporarily) inaccessible. Indeed, as described more fully below, all of the internet providers whose conduct we examined in our research have legally retained the right to moderate content through conditions of their terms of service and acceptable use policies. 

While the results of content moderation decisions by nonplatform enterprises can be observed, very little is known about how or for what reasons companies make their decisions. This systematic lack of transparency is problematic. Policymakers cannot conduct a rigorous examination of the full scope of harms online—including off-platform harms—without transparency. Lacking information, one cannot appropriately assess if, when, how, and according to what standards companies in the ecosystem can and do exert content control. Our goal here is to call out publicly one aspect of the lack of infrastructure transparency (the lack of reporting) and to call for greater information. 

To be clear, “transparency” connotes a broad range of efforts, activities, objectives, and results. In essence, the question that transparency seeks to answer is: To what extent can the organization enhance the public’s understanding of that organization’s mission, policies, authorities, compliance, activities, programs, and so on? When applied to a specific topic (such as content moderation), the question is: To what extent can the organization provide the public with the information it needs to understand what the organization is doing in terms of that particular issue?

One subset of organizational transparency is the idea of transparency reporting. That is, one way to provide some transparency is through periodic reports. Historically, transparency reports by internet enterprises have tended to focus on requests received from the government for information. This, of course, is no accident, and it responds directly to intense public pressure following the Snowden disclosures of U.S. surveillance activities. The focus of the “public” (in other words, Congress and civil society) has for years been on the extent to which companies were providing information to the government, under what authorities, and such. 

More recently, however, some tech companies have expanded the subject matter of their transparency reporting to include disclosures about content moderation decisions. The tech industry practice of publishing transparency reports, which has become widely adopted since Google published the first ever transparency report in 2010, has helped to shed some light on content moderation practices. However, this practice has been, at best, inconsistent and incomplete across the internet “stack.” Our research reveals that virtually none of the entities other than the big social platforms disclose anything about their content decisions. This state of affairs is bad for public policy, and the lack of transparency may harm the online information ecosystem.

Methodology

In a 2020 paper, “Widening the Lens on Content Moderation,” colleagues from the Tech, Law & Security Program at the American University Washington College of Law began to define the “online information ecosystem”—an ecosystem that was much broader than the well-known social platforms that host content, such as Twitter or Facebook. 

This broader ecosystem includes:

• Logical services. The services necessary for accessing, browsing, delivering, hosting, and securing information online. These include internet service providers; virtual private network (VPN) operators; domain name system (DNS) operators, including registrars and registries; content delivery networks (CDNs) like Cloudflare and Akamai; cloud service providers (like AWS); web hosting platforms; distributed denial-of-service (DDoS) mitigation services; and web browser systems. 
• Content services. There are many venues for direct content moderation. This category includes well-known social platforms (such as Twitter or Facebook) as well as search engines (like Google) and app stores (like Google Play). 
• Financial services. The ecosystem also includes the financial systems that facilitate monetary exchange (like PayPal), which lies at the core of much of information exchange. 

In the research for this article, we sought to explore content moderation throughout the online information ecosystem. Our research started by establishing a list of the companies that operate within the online information ecosystem. Because the number of companies within the online information ecosystem is vast, we initially focused on the 100 comparatively larger companies in the logical services category. We then accessed publicly available information for enterprises on the list (principally through their websites) to determine whether and when these companies had ever published transparency reports. 

Since 2010, the annual publication of transparency reports has become a more widely adopted industry standard within the tech industry. Transparency reports are public documents created by technology companies to report on external legal requests and/or internal content moderation measures. 

In the end, determining whether a company had published transparency reports was surprisingly difficult. In many cases, companies did not make it easy to access their transparency reports. Transparency reports were sometimes buried in company websites without clear links. They often did not appear in keyword searches on company websites for “Transparency Report.” Links were not always readily available between one report and the next, making it difficult to find every year available. Some links on company websites to their transparency reports were broken and no longer redirected to the reports or allowed them to be downloaded despite appearing as headings. There was also little consistency between companies about where to locate reports on their websites. Some companies had the reports listed under the legal department, some under customer information, others under a privacy heading. Others did not have them listed at all. Additionally, some companies had offloaded their transparency reports from their main sites to side-sites, such as company blogs. 

Our challenge was compounded by our realization that there is also significant inconsistency in the companies’ approaches to transparency reporting and the conceptions of what it entails. The focus of our research is how transparency reporting can be used as a tool for accessing internal information on how tech companies moderate content. 

However, for many tech companies, this is not what transparency reporting entails. Instead, these companies provide transparency on the extent of their disclosures to law enforcement. Indeed, our examination suggests that many tech companies use the term “transparency report” interchangeably with “law enforcement disclosure report” or “information request reports.” This characterization, paired with the fact that transparency reports are often difficult to locate, makes it difficult to determine which companies have created transparency reports relating to content moderation. 

We examined as many companies as we could identify that had published transparency reports. To complete our analysis, we then evaluated the companies to determine if they fit our definition of companies within the online information ecosystem. What follows is, we believe, the first methodical examination of the extent to which nonplatform companies within the online information ecosystem publish transparency reports on their content moderation decisions.

Findings and Analysis

Over the course of the study, we examined 116 different nonplatform enterprises. (Appendix A includes a full list.) Based on our examination, we concluded that fewer than half of the nonplatform companies have ever published a transparency report in any form whatsoever. Of those that have published transparency reports, most focused exclusively on law enforcement disclosures. In the end, only 10 nonplatform companies (less than 10 percent of the total examined) have published reports that provide data on their content moderation decisions. 

We found 56 companies that are a part of the non-social-platform, online information ecosystem system that had published at least one transparency report (or its equivalent) and 60 similar companies that had never published a transparency report. This finding, as to the infrequency of reporting within the information ecosystem, is remarkable because the structure of our inquiry was intended to over-sample enterprises that we thought were likely to have published transparency reports. Though our supposition—that transparency reports would be more common in larger enterprises than in smaller ones—may be imprecise, we nonetheless believe that our efforts were well targeted. Thus, finding so few transparency reports is a surprising result, and it suggests that the lack of transparency reporting across the ecosystem is greater than we anticipated.

Within those 56 companies that did create transparency reports, the growth in the volume of reporting has been relatively consistent. Since Google’s first transparency report was issued in 2010, the number of publicly available transparency reports has increased steadily over time. As shown in the adjacent chart, the volume peaked in 2020, when 44 reports were available, and the number has since plateaued. (Of course, the number for 2022 is year-to-date and understandably low.)

Of the 56 companies we analyzed that provided transparency reporting, the majority provided little to no information on content moderation policies or procedures. These 56 companies can be broken down into five distinct groups, as shown in the pie chart below.

First, 37 companies (66 percent of the total providing transparency reports) produced only what we characterize as “raw law enforcement reports.” These reports made no mention of content moderation and disclosed only information and data on legal requests and information disclosures. The reports provided information about topics such as the number of civil and criminal information disclosure requests made by law enforcement, the Digital Millennium Copyright Act and copyright legal notices, and in some cases, legal requests demanding the takedown of information.

Within this subcategory, some companies reported only the gross number of requests that were made. Others differentiated the types of requests made by, for example, categorizing them as civil versus criminal or distinguishing requests made by different agencies. Some reports provided information on the company’s response to these requests. They provided, for example, gross totals of the number of requests with which they had complied. Other reports gave detailed information on why and in what number they did not comply with such requests. 

Of the 37 reports in this group, none provided information about the company’s internal policies and how they had been applied. Instead, the reports tended to provide transparency about external actors and the legal entities making the request. While providing additional transparency on governmental requests to tech companies for information on private citizens’ information is beneficial for increased transparency as a whole, that utility is limited when none of these reports shed light on the companies’ internal policies.

A second, smaller group of four companies also provided law enforcement transparency data. These, however, were somewhat more complete reports as they did refer generally to the companies’ own internal policies. However, the reports did so only to the extent that they provided an explanation of the policies concerning company-initiated notices to legal entities.

An additional five companies issued transparency reports that discussed, at length, their broad commitment to the freedom of expression but gave little to no insight into their internal processes to moderate content. Nor did they provide any data regarding their content moderation decisions. One company did acknowledge that it engaged in content moderation when content was found to be in violation of its terms of service and acceptable use policy, but it did not give any specific information about how content moderation decisions were made or the number or type of actions taken, nor did the company provide information about what made the content that was moderated objectionable or worthy of moderation. 

Counting these three subcategories together, we found that 46 out of the 56 companies that provided transparency reports at all (82 percent of the total transparency reports) did so without any meaningful disclosure about their content moderation practices. Their focus was either exclusively on law enforcement disclosures or, in a few instances, on broad hortatory statements about commitment to free expression. 

Only 10 out of the 56 companies that produced transparency reports (and out of the total 116 companies we examined) provided any substantial information regarding their internal processes and content moderation. These 10 companies represent only approximately 18 percent of the total transparency reports available from the companies within the online information ecosystem. Indeed, they represent only approximately 8.6 percent of all companies we examined within the ecosystem.

Even that number overstates the case: Only four enterprises provided detailed breakdowns on the type of content that required moderation and what the consequences of posting harmful or objectionable content were. These companies were Discord (United States), Snapchat (United States), LINE Corporation (South Korea), and Mega (New Zealand). 

Notably, three of these companies are based primarily on social messaging applications. Given that social media platforms and social messaging devices share a similar purpose and structure, it’s unsurprising that these companies were among those that adopted “best practices” for transparency reporting for content moderation. Conversely, this circumstance also reinforces the perception that, at this time, transparency reporting about content moderation is confined narrowly to the social media space and has yet to be propagated in the wider system of companies in the online information ecosystem. The remaining six companies provided content moderation data only, with no explanation of their practices.

Our final finding relates to the growth and plateauing of transparency reporting as a phenomenon. While transparency reporting has become a normalized practice within the technology industry since its emergence in 2010, it is still not practiced widely throughout the industry. Indeed, the adoption of transparency reporting within the industry has seemingly plateaued. Transparency reporting, by number of reports published per year, saw its largest growth in 2013 and 2014. There was no growth between 2015 and 2016 or between 2017 and 2018. 

While the number of companies publishing transparency reports has generally increased since 2010, the number peaked in 2020 with 44 companies publishing reports. The overall number has since decreased slightly to 41 in 2021. We observed that many companies that historically published transparency reports have abandoned the practice. Indeed, seven companies abandoned the practice just this past year, publishing their last reports in 2020. With little interest within the broader information ecosystem and with little public outcry, it seems inevitable that more companies will discontinue the practice of transparency reporting. 

Perhaps even more ominously, some companies that stopped transparency reporting have removed previously existing transparency reports from their websites. Other companies that continue to provide transparency reports maintain only the most recent annual reports on their websites and have removed older reports from their websites. (We were able to verify which companies removed existing transparency reports by comparing our list of collected transparency reports with year lists of transparency reports from the Transparency Reporting Index and through the presence of third-party media and news sites that still had articles available about the publication of transparency reports that no longer exist on their respective company websites.) This practice, especially for companies that continue to participate in transparency reporting, is counterintuitive and contrary to the overall objective of achieving greater transparency and accountability.

Discussion

All of our research and findings speak to a widespread and problematic disinterest in transparency reporting relating to content moderation within the online information ecosystem. The vast majority of companies that operate within the ecosystem have failed to adopt transparency reporting even though the practice is more than a decade old. This is not because they do not make content moderation decisions, but because they choose not to be transparent about their internal processes regarding content moderation. 

Meanwhile, the vast majority of companies that do provide transparency reports remain opaque regarding content moderation or anything to do with their own internal processes and procedures. Instead, they are fully transparent only about systems external to their own that act upon the enterprise. Though many of these companies claim to be champions of transparency, they often decline to disclose their internal procedures. The difficulty in locating transparency reports drives home the reality that most technology companies want the reputational benefits of publishing transparency reports without actually being transparent. 

It is unclear exactly why there is such disinterest in transparency reporting. One plausible reason for the disinterest might be the cost of transparency. Because transparency reporting is only an industry standard and not required by law, the time, money, personnel, and resources needed to publish transparency reports are voluntary costs incurred. For smaller companies, the calculus weighs against expending valuable resources on a voluntary standard that is not widely practiced throughout the industry. For larger companies that engage in a greater amount of content moderation, the higher costs associated with regularizing reporting are likely a factor in their decision to refrain. It is also probable that as the need for content moderation likely skyrocketed following the emergence of the coronavirus, these costs have only grown. This could be a factor in why seven companies ceased publishing transparency reports in 2020. 

Transparency reporting on content moderation also leads to increased criticism and exposure for the companies that engage in the practice. Online content moderation practices are controversial in the media and among policymakers and commentators. Social platforms such as Facebook have come under fire for their content moderation decisions and practices even though Facebook has been one of the more transparent companies regarding their content moderation decisions. Increased transparency and insight into Facebook’s moderation decisions allows for greater insight into its flaws. Public opposition to content moderation has also increased in the past decade (witness the shadow banning debates) and could be a factor in why companies would not want to engage in transparency reporting. 

Increased transparency on content moderation decisions may also bring to light previously unobserved practices by increasing public knowledge of the extent of content moderation in the online information ecosystem. It may be in an enterprise’s best interest to minimize public awareness of just how much content moderation they do. One of the critiques of major social platforms is that they unfairly censor free speech through their content moderation decisions. Other online companies likely want to avoid this criticism and may want to minimize knowledge of the extent of their content moderation powers and usage of this power. Even companies like Truth Social, which make freedom of expression a cornerstone of their services, do not participate in transparency reporting. Perhaps because they want to be seen as champions of freedom of expression, they want to minimize public information on their content moderation decisions to make it appear as though they do not engage in content moderation. 

Conclusion

The results of this survey of transparency reports demonstrate how little is known about content moderation practices throughout the larger online information ecosystem. Simply, more information is needed to understand both the extent and practical application of content moderation policies, as well as the extent of online abuses throughout the online information ecosystem. The lack of information is problematic for policymakers and the public in the ongoing debates around content moderation. As things stand, without transparency reporting, policymakers, the public, and even tech companies are making decisions on these important topics in the dark. Increased transparency reporting benefits everyone, but without a public and/or an industry-level push, transparency reporting will continue to languish.

Appendix A