(Updated: March 2, 2020)
One of the most controversial NSA programs revealed by Edward Snowden was the bulk collection of domestic telephone records under the authority of Section 215 of the USA PATRIOT Act. A detailed analysis of the workings of this program was published on this weblog earlier.
In 2015, Section 215 was replaced by the USA FREEDOM Act, which prohibited the collection in bulk and provided more safeguards. The NSA became much more transparant about this program, which gives the opportunity for the following explanation of how the domestic phone records program currently works.
NSA is also more transparant about things going wrong: last month it revealed that it had to delete all the telephone records collected since 2015 due to technical irregularities.
Screenshot from 60 Minutes from December 15, 2013, showing an NSA contact chaining tool
used for the telephone records collected under Section 215.
Collection under Section 215 USA PATRIOT Act
The NSA started its bulk collection of domestic telephone metadata as part of the President's Surveillance Program (PSP), which president George W. Bush authorized in secret right after the 9/11 attacks. Its purpose was not to spy on random Americans, but to find connections between foreign terrorists and conspirators inside the US.
In May 2006, this bulk collection was brought from the president's authority under that of the FISA Court, based upon a very extensive interpretation of Section 215 of the USA PATRIOT Act. Internally, NSA refers to this kind of collection as BR FISA, with BR for Business Records.
Under Section 215, NSA collected domestic phone records from the three biggest American telecommunication companies: AT&T, Verizon and Sprint. According to government officials, the data provided by these companies consisted mostly of landline phone records, which meant that NSA actually got less than 30% of the total amount of US telephone metadata.
However, as of August 29, 2011, AT&T started to provide cell phone metadata too: ca. 1,1 billion records a day, which would make over 30 billion records each month. Before these records were handed over to NSA, AT&T stripped off the location data, to comply with the FISA Court orders that don't allow the collection of location data. Verizon was apparently not able or not willing to strip the location metadata, so their cell phone records could not be acquired by NSA.
To put these numbers in perspective: with a wireless communications market share of 32% for AT&T, the total number of cell phone metadata for the US would equal roughly 94 billion a month. During the first half of 2012, the NSA's total collection of foreign telephone metadata was 135 billion records a month. In January 2013, mobile phone calls in the Netherlands generated some 7.65 billion records a month.
At NSA, the domestic phone records were forwarded to MAINWAY, which is a centralized system for "contact chaining to identify targets of interest." MAINWAY not only contains domestic telephone metadata, but also foreign telephone and internet metadata, collected both inside and outside the US. Putting both foreign and domestic metadata in one system, allows finding as many connections as possible.
See for more:
- How NSA contact chaining combines domestic and foreign phone records
- Section 215 bulk telephone records and the MAINWAY database
Collection under the USA FREEDOM Act
Because the bulk collection under Section 215 was often regarded unconstitutional, the program was terminated as of November 2015 and replaced by the USA FREEDOM Act (USAFA or UFA), which was incorporated in Title V of the Foreign Intelligence Surveillance Act (FISA). Under this new authority, bulk collection of domestic phone records is not allowed anymore.
Instead, NSA can request only those records that contain phone numbers that have been in contact with an approved "seed" number. This means that all the American telecoms now have to hand over the matching results from both landline and cellphone calls, so it's a much larger pool compared to the situation under Section 215.
How this current domestic phone records program works is explained in remarkable detail in the transparancy report of the NSA Civil Liberties and Privacy Office (CLPO) from January 2016, as well as in the Annual Statistical Transparancy Report from the Office of the Director of National Intelligence (ODNI).
The statistical report for 2017 was published last April and also contains a lot of information about traditional FISA and Section 702 FAA (PRISM and Upstream) collection.
Overview of NSA's collection of domestic phone records under the USA FREEDOM Act
(source: NSA Transparancy Report - click to enlarge)
Seed numbers
The process starts with selecting specific targets and the phone numbers ("selectors") they use. Through the FBI and the Department of Justice, these selectors are submitted to the FISA Court (FISC), which determines whether there's a Reasonable, Articulable Suspicion (RAS) that these numbers are associated with foreign intelligence agents or people engaged in international terrorism. Under Section 215, the RAS was determined by one of 22 designated NSA officials.
After the FISC has approved these numbers, it issues individual orders approving the submission of these specific selectors to the telecommunications providers, and directing those providers to hand over the associated metadata to the proper government agency. According to the ODNI statistical report for 2017, the FISC issued orders for 42 targets in 2016 and for 40 targets last year.
The report doesn't mention the total number of selectors used by these targets. It's these selectors, phone numbers and maybe similar identifiers, that NSA uses as a "seed" to start creating a so-called contact chain. For earlier years, the total numbers of seed selectors were as follows (it's not known how many of these belonged to Americans):
Business records
At NSA, the RAS-approved selectors are entered into what is publicly called the "Enterprise Architecture", but which actually must be the MAINWAY contact chaining system. This returns any selectors from NSA's existing metadata collection that have been in direct contact with the RAS-approved seed selector.
Both the RAS-approved seed selectors and the connected ones from NSA's existing collection are then submitted to the telecommunications providers. They will query their databases of business records for those that contain any of the submitted phone numbers. The results are returned to the NSA, which lets them pass various validation steps, applies data tags and forwards them to the MAINWAY system.
Because a FISC order is valid for up to 180 days, the selectors can be submitted multiple times during that period in order to caputure any new matching records. These business records, or Call Detail Records (CDRs) are defined as "session identifying information" and include:
- Originating telephone numberNSA is not allowed to receive the content of any communication, the name, address, or financial information of a subscriber or customer, or the cell site location or Global Positioning System (GPS) coordinates.
- Terminating telephone number
- International Mobile Subscriber Identity (IMSI) number
- International Mobile Station Equipment Identity (IMEI) number
- Telephone calling card number
- Time and duration of a call
Contact chaining
The ODNI statistical transparancy report from April has a nice graphic that shows how to count the number of business records that the telecoms provide to the NSA:
Example of contact chaining of telephone metadata under the USA FREEDOM Act
(source: ODNI Transparancy Report - click to enlarge)
We see that the RAS-approved seed phone (number) can be in direct contact with a certain number of other phones, which is called the "first hop". Additionally, the providers also have to look for the phones that have been in contact with those first hop phones. This step is called the "second hop". A third hop is prohibited by law, but NSA analysts also determined that a third step is not analytically useful.
This way of contact chaining by linking phone numbers that have been in contact with each other may already be familiar from the reportings about the Section 215 program.
But the graphic also shows something that was rarely made clear: the business records collected by NSA are not just the phone numbers. Two phone numbers that have been in contact with eachother will usually have done so more than once (except for so-called "burner phones" that are intentionally used for one call only).
So for each pair of phone numbers, there can be a lot of records, at least one record generated per phone call or text message, both for the person calling and the person called. The example in the graphic shows 7 phones that produce 6000 call detail records (CDRs) during a certain period of time. This is something to keep in mind when it comes to the huge numbers of metadata collected by NSA.
Number of records
The ODNI transparancy report also provides the real numbers of telephone records collected by NSA under the authority of the USA FREEDOM Act. Although NSA is required by law to provide the annual number of "unique identifiers", the agency doesn't has the technical ability to isolate these unique identifiers within records received from the providers. This means that every single record is counted, even if the same record is received multiple times from one or multiple providers.
The report also explicitly says that the results of contact chaining will likely include both foreign and domestic phone numbers: "while the records are received from domestic communications service providers, the records received are for domestic and foreign numbers." Also, the targeted seed number could be a foreign number, which in the first hop could have called a foreign number, that in its turn could have called another foreign number in the second hop.
With that in mind, the report says that in 2016, the telecommunications providers handed over 151.230.968 phone records to NSA. In 2017 they did so for 534.396.285 records, which is not only a dramatic increase compared to the previous year, but also a probably unexpectedly high number for the just 40 targets approved by the FISA Court.
However, if each of these 40 targets called 50 numbers, and those numbers were also in contact with 50 numbers, we get some 100.000 phone numbers. Let's assume each pair of numbers was involved in 500 calls (or text messages), we already have 50.000.000 records. And this is still without duplicate records, like from multiple providers or recurring requests.
The large increase compared to 2016 may have been caused by a variety of factors, according to Alex Joel, ODNI's chief civil liberties officer: changes in the amount of historical data companies are choosing to keep; the number of phone accounts used by each target and changes to how the telecommunications industry creates records based on constantly shifting technology and practices.
Retention
These domestic call detail records may not be stored for more than 5 years after they were initially delivered to NSA. In addition, the minimization procedures require NSA to destroy promptly any records that are determined not to contain foreign intelligence information. Phone records that have been "the basis of a properly approved dissemination of foreign intelligence information" may be retained by NSA indefinitely.
After these records have been received and stored, they may also be queried, including using search terms associated with US persons. In 2016, NSA used ca. 22.360 search terms for such queries, while in 2017 that number had risen to 31.196.
Deletion
Recently, it turned out that the practical implementation of the collection of domestic phone records under the USA FREEDOM Act is apparently not that easy: in a remarkable public statement from June 28, 2018, NSA revealed that several months earlier, "analysts noted technical irregularities in some data received from telecommunications service providers."
These irregularities occurred in a number of Call Detail Records (CDRs), which meant that NSA was not legally authorized to receive them in that form. It appeared infeasible to identify and isolate the properly produced data, so NSA concluded that it should not use any of these records.
Subsequently, the agency began deleting all the phone records they had acquired since 2015. According to the statement, NSA meanwhile addressed the root cause of the problem for future CDR acquisitions. Civil liberties blogger emptywheel suggests that the records may have contained content or location data, but NSA spokesman Chris Augustine said that the problem did not result in any collection of location records from cellphone towers.
According to the NSA's general counsel, Glenn S. Gerstell, the irregularities were caused by one or more providers who sent NSA data sets that also included some numbers of people the targets had not been in contact with. When the agency then fed those phone numbers back to the telecoms to get the "second hop" records, NSA acquired metadata of people with no connection to the approved targets.
Senator Ron Wyden, a longtime NSA critic who for years tried to get the Section 215 program disclosed, now blamed the providers instead of NSA for the technical problems: "Telecom companies hold vast amounts of private data on Americans," Wyden said. "This incident shows these companies acted with unacceptable carelessness, and failed to comply with the law when they shared customers’ sensitive data with the government."
Former assistant attorney general for national security David Kris said that these "errors illustrated how new problems can sometimes crop up when the government makes systems more complex in an effort to better balance security and privacy."
Speculations
In the public statement it is said that the massive metadata deletion follows from the NSA's "core values of respect for the law, accountability, integrity, and transparency" but outsiders speculated about other motives: were these records destroyed before the Trump administration could misuse them? President Trump also tweeted about this issue and saw it as part of the "Witch Hunt" against him:
Wow! The NSA has deleted 685 million phone calls and text messages. Privacy violations? They blame technical irregularities. Such a disgrace. The Witch Hunt continues!
— Donald J. Trump (@realDonaldTrump) 3 juli 2018
David Kris, former assistant attorney general for national security, replied to Trump that "This NSA program is only for international terrorism, not spying or clandestine intelligence activity, so unless your collusion included terrorism, it should be no problem for you personally!"
UPDATES:
In early 2019, NSA suspended its collection of domestic phone records under the USA Freedom Act "after balancing the program’s relative intelligence value, associated costs, and compliance and data-integrity concerns caused by the unique complexities of using these provider-generated business records for intelligence purposes."
In February 2020, the Privacy and Civil Liberties Oversight Board (PCLOB) issued a report about the NSA's metadata collection under the USA Freedom Act. The Board considered the program "constitutional under settled Supreme Court precedent" and found "no abuse of the program; nor did it find any instance in which government officials intentionally sought records that they knew were statutorily prohibited." During its four years of operation, the program had cost 100 million USD (part of which was paid to the telecom providers). Metadata from the program were used in 15 intelligence reports, only two of which provided the FBI with unique information.
Links and sources
- Privacy and Civil Liberties Oversight Board: Report on the Government's Use of the Call Detail Records Program Under the USA Freedom Act (2020)
- TheMarketsWork.com: A Strange & Unsettling Day (2018)
- NYTimes.com: N.S.A. Purges Hundreds of Millions of Call and Text Records (2018)
- Emptywheel.net: AT&T Pulled Cell Location for its "Mobility Cell Data" (2015)
- HuffingtonPost.com: The NSA’s Telephone Metadata Program Is Unconstitutional (2014)