Jonathan Vanian writes: The CIA’s top techie explained why the intelligence agency is interested in the same big data tech that businesses love.

Doug Wolfe, the CIA’s chief information officer, made an unusual sales pitch to Silicon Valley on Tuesday by arguing that the spy agency and the tech industry have a lot in common.

“Remember, a lot of the solutions we need are similar to the private sector,” he told the crowd at a tech conference in San Francisco, using some tech industry jargon in the process.

Wolfe, a 30-year veteran of the CIA, was trying to explain the intelligence agency’s interest in a hot technology for data-processing called Spark that’s the current rage for big data nerds. It lets businesses sift and analyze data much quicker than they could just a decade ago.

Of course, Wolfe stayed silent about why the CIA is so interested in processing huge amounts of data. But the reason was hardly a mystery. As part of its spy mission, the CIA invariably wants to quickly glean insights from huge troves of information it and fellow spooks at the National Security Agency collect on a daily basis. Records of international money transfers, cellphone calls, and biometric data about possible terrorists are just some of the inevitable areas of interest.

It’s the same kind of technology that many big businesses use everyday. It turns out that covert CIA operations and routine online marketing campaigns by retailers are much the same. [Continue reading…]