The Latest NSA Documents V: the NSA Investigates Its Metadata Compliance Problems, Takes Remedial Steps, and Reports Back to the FISC

The latest installment in the NSA declassification story comprises five documents.  The first is an internal NSA compliance review; the second is a court filing regarding that review.  The latter also refers to three sworn statements, two from the NSA director and one from the FBI director.  All concern the NSA's compliance with FISC orders regarding the collection, querying, and dissemination of telephony metadata.

The "End to End" Review

Our first and perhaps most significant item is the so-called "End-to-End" review.  As NSA Director Keith Alexander had indicated in an earlier declaration to the FISC, an internal NSA outfit---its Business Records FISA Compliance Review Team---had undertaken a thorough study of the agency's "instrumentation and implementation" of FISC authorizations regarding telephony metadata.  Upon completion, that effort, as well as its findings and recommendations, were summarized in a 46-page document dated June 25, 2009 ("End-to-End Review").   (As we explain below, the document's key findings were later shared with the FISC, in an August report.)

The End-to-End Review has five parts: first, an executive summary; second, a brisk treatment of the results of NSA's analysis, in both identified areas of concern to the FISC and newly discovered problem areas; third, an explication of the processes employed during the End-to-End Review and of the metadata program's workings; fourth, an overview of NSA minimization and oversight protocols; and fifth, a sketch of "future architecture" for metadata collection, assuming that the FISC ultimately will agree once more to authorize NSA unilaterally to query stores of collected metadata.  (It did so later---but that's getting ahead of the story).

The gist of the executive summary, and indeed, of the entire Review and some related, subsequent court filings, is to acknowledge instances of good-faith, inadvertent non-compliance, while identifying remedial measures for the future.  NSA depicts an apparent disconnect between lawyers who sought FISC approval for telephony metadata collection, and engineers who designed systems to handle the metadata.  That disconnect was not the product of malice.  Instead, NSA characterizes the problem as  "a basic lack of shared understanding among the key mission, technology, legal and oversight stakeholders."  This was, the executive summary continues, only worsened by the extraordinary complexity associated with the metadata program, and by the historical focus of NSA's internal oversight mechanisms.  The latter trained primarily----and, NSA seems to imply, inordinately---on how analysts accessed the metadata.

Areas of Concern

Overview out of the way, the Review turns to "detailed analysis" of the problems it uncovered, with each problem getting a brief (usually less than a page) "description," followed by an equally brief recitation of implemented or in-progress "remedial steps."  Critically, NSA not only elaborates on compliance incidents it previously reported to the FISC---like the establishment of an "alert list" comprised of numbers that had not been deemed to satisfy a court-imposed "reasonable and articulable suspicion" standard, for the querying of metadata.  On top of this, NSA's inquiry also identifies nine new varieties of violations.  By way of examples, the newly uncovered shortcomings, and NSA's proposed remedies (completed and ongoing), include:

• NSA's repository for individual metadata "one hop" chains.  The End-to-End Review's second part says that NSA's compliance department was not aware, until January 2009, of this repository's "existence in the technical architecture."  For that reason, it had not been audited previously---though an audit eventually was conducted in February 2009.  Since then, and with the addition of "enhanced auditing capability," NSA has "found no evidence of improper queries"---meaning, presumably, no evidence of queries that were not supported by reasonable suspicion.  To be sure, such auditing has been somewhat limited, as the relevant NSA system suffered a crash in 2008.  Nevertheless, the End-to-End Review says NSA recovered "sufficient data to permit [compliance personnel] to conduct sample audits of queries since the [FISC's initial order authorizing bulk metadata collection]."

• The activities of NSA's "Data Integrity Analysts."  Redactions frustrate clarity here, but we can glean that data integrity personnel---folks tasked with ensuring the proper formatting and storage of collected metadata---had a practice of "populating [REDACTED] numbers in NSA databases outside the [telephony metadata] databases."  That not-so-clear-to-the-reader-because-of-the-redactions habit was not disclosed to the FISC.  And it became an area of "concern" for NSA, in May of 2009.  To address it, the End-to-End Review says NSA took steps to limit data integrity analysts' contact, by quarantining metadata-derived identifiers, and by shutting off access to a file containing a number of such identifiers.

• Use of correlated selectors to query the metadata.  A "selector" is, in the techno-parlance employed by the End-to-End Review, a "communications address."  This can be considered "correlated with other communications addresses when each additional address is shown to identify the same communicant (s) as the original address[.]"  The trouble?  Initially, "if there was a successful [finding of reasonable suspicion] made on any one if the selectors in the correlation, all were considered RAS [reasonable suspicion]-approved for purposes of a [metadata] query[.]"   The solution?  NSA suspended the automated system that furnished correlation results to metadata analysts, and ended the practice of treating correlations as RAS-approved across the board.

• Internal Sharing of Metadata.   Here, NSA suggests that it might not have erred at all, in that it had interpreted past FISC guidance as not prohibiting internal, NSA-only sharing of metadata queries.  It seems a small team of specially cleared metadata analysts would pass on "unminimized query results" to colleagues focused on other NSA counterterrorism activities---which was perfectly consistent with FISC decisions, in NSA's view.  Nevertheless, the End-to-End Review says NSA began, in June of 2009, to limit "access to unminimized [metadata] query results to only authorized analysts."  The Review also observes that the FISC eventually blessed internal sharing, provided that analysts receving query results--be they metadata people or people working on other NSA projects---received appropriate training and guidance on FISC rules.

• External Access to Unminimized Metadata Query Results.  NSA had internal sharing problems as well as external sharing problems, the latter involving an NSA counterterrorism database that approximately 200 CIA, FBI and NCTC analysts could access.  The fix here was technical, and involved (among other things) the disabling of an external hyperlink, and termination of other agencies' account access.  According to the Review, an audit of how many times outside people accessed the metadata, and what they accessed, is ongoing.

Also in the Review's catalog of newfound goofs: the access of system developers to metadata, while testing new system tools; the inappropriate production, by a telecommunications provider, of "foreign-to-foreign" metadata to NSA; the designation, without prior approval of NSA lawyers, of a "few" (less than ten, going by initial NSA estimates) domestic telephone numbers as RAS-approved; and the dissemination of U.S. person-identifying information in NSA metadata reports without prior approval--as required by the FISC---from NSA's Chief of Information Sharing Services.

The Review's Processes, and the Metadata Program's Workings

The End-to-End Review's third part is about process---that is, what NSA did, in order to determine the extent of its compliance (or not) with FISC metadata orders.  The agency's testing was "repeatable and well-documented," and included, among other things: a review of legal requirements, a mapping of the functioning, step-by-step, of the metadata program. The agency then describes its "findings"---which read like a hyper-technical tour through the NSA's metadata activities.  These touch, seemingly, on almost every facet of the program: the receipt of metadata from providers, and its sorting and labeling; the allocation of incoming metadata by NSA's "corporate file forwarding service";  the input of metadata into NSA's contact chaining system (and its enhanced security, owing to recent NSA improvements); NSA's use of a special tool to "view detailed data about specific calling events;" the comparison of incoming metadata to certain telephone numbers, for "alert list" purposes; the way telephone identifiers are assessed in light of the "reasonable, articulable suspicion" standard; the decision and reporting processes employed by metadata analysts; and so on.

Minimization and Oversight

The fourth part of the End-to-End Review discusses minimization procedures and oversight mechanisms---with a mind towards enhancing NSA capabilities in auditing, documentation, and training. Minimization-wise, the End-to-End Review says NSA analysts must complete training and pass a test on USSID minimization procedures every two years.  They also receive compliance briefings from OGC attorneys.  Oversight-wise, the Review here mentions 13 audits of queries of the metadata program, conducted in connection with standing requirements or as a consequence of particular compliance incidents.  Regular audits are also conducted on a weekly basis, too, according to the Review.  There also are oversight documents.  These variously describe spot checks by NSA's general counsel's office, specific audit procedures, and the identification of particular compliance problems, among other things.   Finally, the End-to-End Review describes various training programs, both old and more recent, for metadata handling.

Future Architecture

In its fifth part, the End-to-End Review proposes further technical safeguards, should the Court permit NSA to return to automated access to metadata.  The agency foresees an augmentation of the alerting process; migration to a next-generation system for dataflow and life cycle management; new access control applications; standardization of metadata, so as to expedite data purging along the FISC's timeline; a more efficient target database; greater capabilities to conduct call chaining; and enhanced auditing.

The Executive Branch Reports Back to the FISC

Its forensic work done, NSA next complied with a prior FISC order---by informing the FISC of the End-to-End Review's findings, the value of the metadata program to national security, and the agency's path forward.  It did so by means of this 69-page, August 19 report ("August Report"), which bore the signature of then-Assistant Attorney General David Kris, of DOJ's National Security Division.  (Attached to the filing were two declarations by NSA Director Keith Alexander, dated August 3 and 19, 2009, respectively, and an affidavit by then-FBI Director Robert S. Mueller, dated August 13, 2009.  All three attachments are described further below.)

The FISC had ordered the executive branch, among other things, to explain the metadata collection effort's significance for national security.  After quickly overviewing the relevant background, the August Report does that.   Predictably, the document emphasizes that, quite unlike other forms of telephone surveillance, bulk telephony metadata collection allows for contact chaining.  It thus greatly enhances NSA's ability to uncover historical as well as current behavior patterns by suspected terrorists.  The August Report also urges the Court to consider a potent counterfactual, regarding a communication between one of the 9/11 hijackers, in California, and an al Qaeda safehouse in Yemen:

[]NSA intercepted and transcribed seven calls made by [9/11] hijacker Khalid al-Mihdhar, then living in San Diego, California, to a telephone identifier associated with an al Qaeda safehouse in Yemen . . . NSA intercepted these calls through its overseas SIGINT collection and, as noted above for telephone calls originating within the United States, the calling party identifier was not included in the signaling information. . . Because they lacked the U.S. telephone identifier and had nothing in the content of the calls to suggest that al-Mihdhar was inside the United States, NSA analysts mistakenly concluded that al-Mihdhar remained overseas when, in fact, he was in San Diego. . .The BR metadata, by contrast, would have included the missing information and might have permitted NSA analysts to place al-Mihdhar within the United States prior to the attacks and tip that information to the FBI.

With that, the August Report turns to the End-to-End Review's findings (which you can review in greater detail above), and NSA's implementation of remedial measures.  Importantly, NSA says that all non-compliance events have been addressed.  The Report divides those events into two categories: unauthorized queries using automated systems and operator errors.  To handle the former, explains the August Report, NSA essentially turned off the offending systems---among them a "Telephony Activity Detection Process" and another process, the name of which has been redacted.  NSA also opted for a new system, known as “Emphatic Access Restriction” ("EAR") which limits access to queries using an RAS-approved identifier.  As for the latter, installation of EAR also would help prevent operator error, too.  So would two more safeguards recently implemented and described by the August Report---designating unapproved identifiers as such, and increasing oversight and training of persons with access to the metadata.

The Report turns then to other matters identified during the End-to-End Review: the production of foreign-to-foreign records, which NSA stopped after the May 29, 2009 order; and its handling of credit card information collected incidentally, along with associated telephony metadata.  As to the second of the pair, the Report explains that NSA strips any credit card information from records on which such information is found---and that it is linked to only a "small percentage” of telephony records produced to NSA.  There are some older records, for which stripping credit card information is not an option.  In such cases, NSA stores the data in secure locations.

NSA's August filing then concludes, by overviewing reforms instituted by the agency.  The August Report says NSA installed additional IT systems, further trained analysts, increased communications between NSA and the Department of Justice, and added the position of a Director of Compliance---whose sole responsibility is, unsurprisingly enough, to ensure the agency's compliance with FISC orders.

In its final pages, the August Report sets forth the government’s plan to request the FISC’s permission to resume querying its metadata, in part because of the safeguards described above.  And to bolster its position, NSA proposes some further, add-on reforms, ones that the agency presumably would implement upon restoration of unilateral query capabilities: a review of RAS determinations every 180 days for U.S. identifiers, and alerting of NSA analysts of the time period for which a telephone identifier has been associated with FISA-targets.  The Department of Justice also proposes to review NSA queries, and to report on those queries to the FISC, twice every ninety days.

NSA Director Keith Alexander August 3 Declaration

Attached to the August Report are two declarations by NSA Head Keith Alexander (dated August 3 and 17, 2009), which demonstrate the value of the telephony metadata program. In the former, Alexander articulates two specific reasons for continued access to telephony metadata.  This, he says, allows the government to analyze past connections and patterns, and helps NSA analysts understand the “communications tradecraft of terrorist operatives."  Alexander notes that NSA issued 277 reports to the FBI and other intelligence customers between May 2006 and May 2009.  All of those, he avers, were derived from NSA-collected telephony metadata.

FBI Director Robert S. Mueller August 13 Affidavit

FBI Director Robert S. Mueller's affidavit likewise touts the metadata program's value for his agency’s operations.  The program allows the FBI to “connect the dots” they already have, and to identify new dots---and thus new investigations.  Four FBI investigations have been re-opened and/or significantly strengthened by metadata provided by NSA, according to Mueller.  Building on that point, Mueller's affidavit offers a chart (recreated below) laying out the impact of the metadata on investigations in 2006, 2007 and 2008:

* Because certain IIRs were issued to multiple countries, the FBI issued a total of 51 IIRs to foreign partners.

NSA Director Keith Alexander August 17 Declaration

The NSA chief's second declaration delves into details of the End-To-End Review, discusses steps NSA has taken to resolve compliance issues identified in the review, and recommends additional measures it would implement---if the FISC elects to allow NSA to resume unilateral metadata querying.