Robert X. Cringely writes: The latest Edward Snowden bombshell that the National Security Agency has been hacking foreign Google and Yahoo data centers is particularly disturbing. Plenty has been written about it so I normally wouldn’t comment except that the general press has, I think, too shallow an understanding of the technology involved. The hack is even more insidious than they know.
The superficial story is in the NSA slide (above) that you’ve probably seen already. The major point being that somehow the NSA — probably through the GCHQ in Britain — is grabbing virtually all Google non-spider web traffic from the Google Front End Servers, because that’s where the SSL encryption is decoded.
Yahoo has no such encryption.
The major point being missed, I think, by the general press is how the Google File System and Yahoo’s Hadoop Distributed File System play into this story. Both of these Big Data file systems are functionally similar. Google refers to its data as being in chunks while Hadoop refers to blocks of data, but they are really similar — large flat databases that are replicated and continuously updated in many locations across the application and across the globe so the exact same data can be searched more or less locally from anywhere on Earth, maintaining at all costs what’s called data coherency.
Data replication, which is there for reasons of both performance and fault tolerance, means that when the GCHQ in London is accessing the Google data center there, they have access to all Google data, not just Google’s UK data or Google’s European data. All Google data for all users no matter where they are is reachable through any Google data center anywhere, thanks to the Google File System.
This knocks a huge hole in the legal safe harbor the NSA has been relying on in its use of data acquired overseas, which assumes that overseas data primarily concerns non-U.S. citizens who aren’t protected by U.S. privacy laws or the FISA Court. The artifice is that by GCHQ grabbing data for the NSA and the NSA presumably grabbing data for GCHQ, both agencies can comply with domestic laws and technically aren’t spying on their own citizens when in fact that’s exactly what they have been doing. [Continue reading…]
