NSA Compliance and Congress’s Plan: How to Account for Flaws in the Metadata Program?

In the extraordinary transparency that followed Edward Snowden’s 2013 revelations, one tantalizing mystery remained: how did the NSA persist until early 2009 in querying metadata under the now-replaced section 215 program with search terms (“identifiers”) that lacked a key requirement imposed by the Foreign Intelligence Surveillance Court (FISC)? John DeLong and Susan Hennessey recently posted a comprehensive explanation of this serious compliance problem. The problem John and Susan recount entailed using identifiers (such as phone numbers) for which analysts lacked a “reasonable and articulable suspicion” (RAS) of links to terrorism. John and Susan’s post provides an invaluable service in establishing that the RAS compliance problem did not stem from willful violations. However, their post does not fully address the disconnect that the RAS compliance problem revealed between the NSA’s institutional culture and Congress’s plan.

Determining whether NSA has remedied this disconnect requires more than a finding that no NSA employee willfully departed from the FISC’s metadata order. Ascertaining whether the NSA cleaned up its act when it joined with the Justice Department in disclosing the problem to the FISC in early 2009 requires understanding three additional concerns: the government’s failure to 1) solicit the FISC’s guidance on a matter that cried out for further explanation; 2) understand that resort to non-RAS-approved identifiers undermined the use restrictions that Congress had expected that the FISC would impose; and, 3) spell out to Congress in plain English what had gone wrong.

The failure to seek FISC guidance suggests that at least until 2009, a troubling gap existed in the interactive, iterative relationship that intelligence community officials have long claimed with the FISA Court (see my article on the FISC and Article III here). That functional relationship is real, as the Privacy and Civil Liberties Oversight Board (PCLOB) noted in its 2014 report on section 702 of the FISA Amendments Act. In the 2009 matter, FISC judge Reggie Walton employed the same robust and resourceful approach to review that the PCLOB praised. However, the RAS problem suggests that intelligence community officials were more reticent about seeking FISC guidance pre-2009, leading Judge Walton to observe that the FISC depended on the government’s ability to promptly recognize its own mistakes and bring them to the court’s attention.

In a fully functional relationship, government officials who weren’t sure pre-2009 whether the RAS standard governed querying of incoming metadata or merely of metadata already stored could have availed themselves of a simple remedy: ask the judge. Lawyers do this all the time, for example in discovery matters in civil litigation that are far less consequential than the parameters of national security surveillance. It’s hard to see why this basic step wasn’t on the radar screen of experienced NSA officials. Cognitive psychologists like Nobel Prize winner Daniel Kahneman (“Thinking Fast and Slow”) would attribute this failure to flaws in human inference that affect us all—find a lawyer who has never made a mistake, and you’ll find a lawyer who has never practiced law. That said, it’s an organization’s job to anticipate those errors and correct for their occurrence, as John wisely noted in his excellent article (see abstract here) on compliance. The RAS problem revealed that NSA took entirely too long to learn this fundamental organizational lesson.

These pre-2009 gaps—which Dr. Kahneman would attribute to imperfect human inference—have a different meaning and import for privacy advocates, precisely because applying the RAS standard to incoming metadata is clearly more privacy-protective than exempting such data from the FISC’s order. Privacy advocates have wanted the NSA to be more proactive in asking whether collection programs are necessary, given their impact on privacy. (See the Brennan Center paper by Liza Goitein and Faiza Patel here.) I worry that some privacy advocates underestimate the real threats that the intelligence community seeks to diffuse, and the procedures agencies like the NSA have put in place to prevent abuses. The RAS problem makes such flawed assessments appear to be right on the mark—that’s one reason this snafu was so profoundly counterproductive.

Moreover, Judge Walton of the FISC clearly viewed the RAS problem as severe: that’s why he required the government through most of 2009 to seek FISC approval for all identifiers used to query metadata. Just as private sector businesses are accountable to regulators like the Securities and Exchange Commission, the NSA and the Justice Department are accountable to the FISC. Getting regulators upset is not a recipe for success in the corporate world and it’s similarly imprudent in government work. Officials should have grasped much earlier that the lower privacy protection implicit in the failure to subject incoming metadata to the RAS standard upset the balance that Congress had empowered the FISC to enforce.

Finally, although the Justice Department and the NSA are to be commended for disclosing the RAS problem to the FISC, the documents constituting disclosure to Congress lack the comprehensiveness of the government’s disclosures to the court. In this 2009 letter to then-House Intelligence Committee chair Silvestre Reyes, a Justice Department official fails to acknowledge the scope and duration of the RAS organizational snafu. It is entirely possible—and even probable—that conversations between intelligence community and House and Senate Intelligence Committee members were more candid. But congressional oversight sometimes requires more than just a conversation with oversight committees. The documentary evidence does not suggest the interactive, reciprocal relationship that diligent legislators had a right to expect. Some legislators, including surveillance critics like Oregon senator Ron Wyden, clearly understood what had occurred. But intelligence community officials would have been better served in 2009 by more comprehensive and straightforward disclosure to Senator Wyden’s congressional colleagues.

John and Susan’s post shows that 2009 was almost as important a year for the NSA as 2013, the year of Snowden’s revelations. Since 2009, the FISC has demanded that the NSA install technology that promotes compliance, through search filters, privacy audits, and other safeguards—and the NSA has complied. Even more importantly, the NSA has devoted human capital to ongoing civil liberties and privacy efforts (which should help induce the more proactive approach to privacy recommended by Margo Schlanger here). That human commitment is the best answer to the flaws in human inference that caused the RAS problem. To reach its full potential, such efforts need to fully reckon with past mistakes. John and Susan’s post is a valuable step in the right direction, but the journey is a work in progress.