New Zealand and XKEYSCORE: not much evidence for mass surveillance

Since March 5, The New Zealand Herald and the website The Intercept published a number of stories based on top secret documents regarding New Zealand. These stories followed last year's claims by Edward Snowden saying that the New Zealand signals intelligence agency GCSB is involved in indiscriminate and illegal mass surveillance of ordinary citizens.

Here we will take a close look at the original documentes that accompanied these reportings and put them in a broader context in order to see whether they support these claims or not. Attention will also be paid to the notorious XKEYSCORE system.

- GCSB satellite collection - XKEYSCORE - GCSB cable access - Conclusion

The listening station at Waihopai (SIGAD: NZC-333) in New Zealand
after activists deflated one of the kevlar radomes in April 2008
(Source: GCSB presentation - Click to enlarge)

 

GCSB satellite collection

In the first story from March 5, it was claimed that New Zealand's signals intelligence agency GCSB conducted "mass spying on friendly nations" in the South Pacific on behalf of the Five Eyes partnership, which consists of the United States, the United Kingdom, Canada, Australia and New Zealand.

The allegation of "mass spying" seems to be based upon an excerpt from an GCHQ wiki page from about 2011, which talks about "full-take collection" at New Zealand's satellite intercept station in Waihopai (codenamed IRONSAND):

Excerpt from a GCHQ wiki page from about 2011 about XKEYSCORE (XKS)
access at the Waihopai satellite station, codenamed IRONSAND
(Click to enlarge)

A GCSB report from July 2009 says that GCSB users were trained by NSA XKEYSCORE trainers "in anticipation of full-take collection and 2nd party sharing" with the full-take collection expected to be running by October 2009.


"Full-take" collection

The New Zealand Herald explained that "full-take collection means the base now collects and retains everything it intercepts: both the content of all the messages and the metadata". If that would be true, then one could probably speak of "mass surveillance".

But later on, the report quotes the German magazine Der Spiegel, which reported already in 2013 that XKEYSCORE "enables 'full-take' of all unfiltered data over a period of several days". The latter is an important detail, but neither The New Zealand Herald, nor The Intercept paid any further attention to it.

When New Zealand's prime minister John Key was asked about the "full-take" at a press conference, he told a reporter: "With the greatest of respect, I don't actually think you understand the technical term and it's not my job to explain it to you". This is the standard response governments give in these matters, rather letting citizens think they are under massive surveillance than explaining what really happens...
 

XKEYSCORE

In the GCHQ wiki entry we also see two check boxes with next to them the Waihopai station mentioned as "GCSB_IRONSAND_WC2_FULL_TAKE". The abbreviation WC2 stands for WEALTHYCLUSTER 2, which is apparently the second generation of a system that is used to process low data rate signals: it sessionizes all of them and then forwards them to XKEYSCORE.

Using WEALTHYCLUSTER processing is called the traditional version of XKEYSCORE, which is used for satellite and terrestrial radio signals. For higher data rates, like on fiber-optic cables, it was/is not possible to forward all data to XKEYSCORE.

These yet unfiltered internet communication sessions forwarded to XKEYSCORE are called the 'full-take'. They are only stored for a short period of time: content is buffered for 3 to 5 days (sometimes shorter or sometimes longer, depending on the amount of traffic), and metadata for up to 30 days. In other words, XKEYSCORE creates a rolling buffer which is continually being rewritten:

Slide with some main characteristisc of the XKEYSCORE system
See also another, similar NSA presentation about XKEYSCORE

This buffering enables analysts to perform federated queries using so-called "soft selectors", like keywords, against the body texts of e-mail and chat messages, digital documents, spreadsheets in English, as well as in Arabic and Chinese. XKEYSCORE also allows analysts to look for the usage of encryption, the use of a VPN or the TOR network, and a number of other things that could lead to a target.

This is particularly useful to trace target's internet activities that are performed anonymous, and therefore cannot be found by just filtering out known e-mail addresses of a target. When such content has been found, the analyst might be able to find new intelligence or new "strong selectors", which can then be used for starting a traditional search.


XKEYSCORE Fingerprints

To use XKEYSCORE more efficient, analysts can create so-called 'fingerprints', which are rules that contain search terms (especially all the correlated identities of a certain target) that are automatically executed by the system. Some examples of XKEYSCORE fingerprints were disclosed by German regional television on July 3, 2014, who presented them as excerpts of XKEYSCORE's source code.

Until now, The New Zealand Herald has published two XKEYSCORE fingerprints that define GCSB targets: one related to candidates for the job of director-general of the World Trade Organisation (WTO), and another one related to the Solomon Islands, for which the fingerprints show that GCSB (and/or NSA) was interested in documents from the government of this island state, as well as in the Truth and Reconciliation Commission and former militia groups.


GCSB targets

Another document disclosed by The New Zealand Herald and The Intercept shows that GCSB also spies on China, Pakistan, India, Iran, South Pacific Island nations (like Tuvalu, Nauru, Kiribati and Samoa, Vanuatu, New Caledonia, Fiji, Tonga and French Polynesia), the diplomatic communications of Japan, North Korea, Vietnam, and South America, as well as French police and nuclear testing activities in New Caledonia, and even on Antarctica.

A number of these targets, and some others, were already listed in a 1985-86 annual report of GCSB (classified as TOP SECRET UMBRA), which was accidently released in 2006. So although it might be embarrassing for the New Zealand government that the spying on nearby friendly island states was exposed, it is nothing new and nothing what is very far out of the range of what intelligence agencies usually do.
 

"Collect it All"

In a GCSB presentation (pdf) about the Waihopai satellite station from April 2010 we read: "To brief IS on the MHS ‘Collect It All’ initiative" - with IS being the abbreviation for IRONSAND, the codename for Waihopai; and MHS for Menwith Hill Station, NSA's large satellite facility in England.

This seems to confirm that "Collect It All" was initially a project for the Menwith Hill Station, maybe meant to be extended to other satellite collection facilities, but not the primary aspiration for NSA's collection efforts in general, as Glenn Greenwald claimed in his book No Place To Hide.*

As evidence, Greenwald presented a slide from a 2011 presentation for the annual Five Eyes conference, but that shows that "Collect it All" actually refers to just one particular stage of the collection process for satellite traffic:

- On top of the diagram, the process starts with receiving the satellite signals ("Sniff it All") and this is followed by "Know it All", which is about detecting (survey) what kind of traffic certain communication channels contain.

- The stage for which they aim "Collect it All" is when signals are processed into usable data by conversion, demodulation and demultiplexing. This is done through systems codenamed ASPHALT and ASPHALT PLUS, but no further information on these system has been published. Apparently "Collect it All" is about increasing the capability to process signals.

- The next stage is "Process it All" where, after a Massive Volume Reduction (MVR) to get rid of useless data, XKEYSCORE (XKS) is used to search for things that are of interest. The last two stages are about analysing data at a large scale and share them with GCHQ and NSA's satellite intercept station in Misawa, Japan.

Photo of what might be XKEYSCORE equipment at the NSA's
European Cryptologic Center (ECC) in Griesheim, Germany
(Source: ECC presentation (pdf) - Click to enlarge)

Targeted collection

Combining the earlier disclosed information about XKEYSCORE shows that neither "full-take", nor "Collect it All" means that "everything" ends up in some NSA database (typically PINWALE for content and MARINA for metadata). This only happens with data that is extracted based upon 'strong selectors', 'fingerprints', or manual searches by analysts when they think it contains valuable foreign intelligence information.

A 2012 NSA document about a training course for XKEYSCORE, published by Der Spiegel in June 2014, says that this system helps analysts to "downsize their gigantic shrimping nets [of traditional collection methods] to tiny goldfish-sized nets and merely dip them into the oceans of data, working smarter and scooping out exactly what they want".

This suggests that XKEYSCORE is able to sort out data in a way that is even more targeted than the traditional method, in which communications are filtered out by internet addresses. This would make XKEYSCORE even less the "mass surveillance tool" as it is called by Snowden.
 


GCSB cable access

Besides the satellite station in Waihopai and the High-Frequency radio intercept facility near Tangimoana, some snippets disclosed in September 2014, show that GCSB also started a cable access program codenamed SPEARGUN, for which the first metadata probe was expected mid-2013. According to The Intercept, this program might be about tapping the Southern Cross cable, which carries "the vast majority of internet traffic between New Zealand and the rest of the world".

A bit confusing is that in a 2012 GCSB presentation (pdf), project SPEARGUN is listed among topics related to the "IRONSAND Mission", but maybe this means that the mission of this satellite intercept station in Waihopai was extended to include cable operations too.

IRONSAND is in the north east of the South Island of New Zealand, while the landing points for the Southern Cross cable are in the north of the North Island, a distance of more than 500 kilometers. It's possible that from the Waihopai station the actual cable intercept facilities are remotely controlled, maybe through a secure Virtual Private Network (VPN) connection over the domestic Aqualink cable:

The access points to the Southern Cross cable could then be identical with the "NSA facilities" in Auckland and "in the north" of the country, which Edward Snowden hinted to in his speech on the "Moment of Truth" meeting in Auckland on September 15, 2014.


Snowden's claims

The Intercept presented this cable access as a "mass metadata surveillance system" capable of "illegal domestic spying" on the communications of New Zealanders. These claims seem to be based upon a rather pathetic statement from Edward Snowden himself:

"If you live in New Zealand, you are being watched. At the NSA I routinely came across the communications of New Zealanders in my work with a mass surveillance tool we share with GCSB, called “XKEYSCORE.” It allows total, granular access to the database of communications collected in the course of mass surveillance. It is not limited to or even used largely for the purposes of cybersecurity, as has been claimed, but is instead used primarily for reading individuals’ private email, text messages, and internet traffic".

Snowden pretends that XKEYSCORE is primarily used to snoop on the communications of private citizens, as if GCSB, NSA and the other partner agencies don't have way too many other targets (see for example the long list of countries targeted by GCSB) and waste their time on ordinary civilians. Snowden however continues:

"The GCSB provides mass surveillance data into XKEYSCORE. They also provide access to the communications of millions of New Zealanders to the NSA at facilities such as the GCSB station at Waihopai"
"It means they have the ability see every website you visit, every text message you send, every call you make, every ticket you purchase, every donation you make, and every book you order online"

This is also misleading, because, as we have already seen, GCSB isn't very much interested in "your" private communications. In his "Moment of Truth" speech, Snowden claimed that he would have been able to enter for example the e-mail address of prime minister John Key in XKEYSCORE to get access to all content and metadata of his internet activities.

What Snowden briefly acknowledged in this speech, but left out in his statement for The Intercept, is that such searches are constrained by policy restrictions. Indeed, every analyst who works with XKEYSCORE and wants to query data collected in New Zealand, has to do a training on the New Zealand Signals Intelligence Directive 7 (NZSID7), which contains the rules about what GCSB is allowed to do.

As GCSB is not allowed to collect communications of New Zealanders (except for when there's a warrant to assist domestic agencies), this means that the other Five Eyes agencies aren't allowed to do that either. Snowden would therefore not have been allowed to look at the communications of prime minister Key.

> See also: Snowden would not have been able to legally "wiretap anyone"

Not only must all queries against data from New Zealand sources be compliant with both the NZSID7 and the Human Rights Act (HRA), they will also be audited by GCSB:

Excerpt from a GCHQ wiki page from about 2011 about XKEYSCORE (XKS)
access at the Waihopai satellite station, codenamed IRONSAND
(Click to enlarge)

Snowden however considers these policy restrictions not sufficient because analysts "aren't really overseen". For GCSB, a 2013 review report found that there were indeed problems with oversight, but the new GCSB law, which is opposed by many people because it would supposedly enable "mass surveillance", actually also strengthens oversight. NSA noticed this too.


The government's response

New Zealand's prime minister John Key rejected the reportings by The New Zealand Herald, saying that "Some of the information was incorrect, some of the information was out of date, some of the assumptions made were just plain wrong". He strongly denied that GCSB collects mass metadata on New Zealanders, but he acknowledged that the agency had tapped into the cable, but only for the purposes of a cybersecurity program codenamed CORTEX.

As a proof, several secret government documents were declassified, but from them it doesn't become clear whether CORTEX really is the same program as the cable access which is codenamed SPEARGUN in the NSA and GCSB documents. According to Key, the CORTEX cybersecurity system was eventually scaled back and now only protects specific entities in the public sector and some private companies.

A snippet from an NSA document says that the implementation of the cable access project SPEARGUN was awaiting the new 2013 GCSB Act. It was said this was because the new law would enable "mass surveillance", but the proposed law also authorizes GCSB to ensure cybersecurity, which would support the statement of the government.

 

Conclusion

As the disclosed documents only contain a few lines and no further details about the cable acces codenamed SPEARGUN, it is not possible to say for sure whether this is about intercepting communications from the Southern Cross cable, like the Snowden-related media claim, or that it is actually a cybersecurity program, like the government says.

What did become clear is that XKEYSCORE isn't really a "mass surveillance tool", but is actually used to collect data in a way that is at least just as targeted as traditional methods. Many of GCSB's targets came out as legitimate, some are more questionable, but none of them included the bulk collection of communications from ordinary citizens, whether domestic or abroad.

Snowden also said that there are "large amounts of indiscriminate metadata about the communication and other online events of citizens" from all Five Eyes countries. But apart from the domestic phone records collected by the NSA, no evidence has yet been presented for such collection in the other countries.



Links and Sources
- EmptyWheel.net: What an XKeyscore Fingerprint Looks Like
- The New Zealand Herald: Bryce Edwards: The ramifications of the spying scandal
- The Press: We're snooping on the Pacific...so what?
- Report: Review of Compliance at the Government Communications Security Bureau (pdf) (2013)
- ArsTechnica.com: Building a panopticon: The evolution of the NSA’s XKeyscore