Pratap Chatterjee writes: We live digital lives now, flitting from Facebook to YouTube, checking our iPhones and BlackBerries, and chatting with our loved ones on Skype. Very few of us worry too much about tweeting our personal opinions on politics or chatting with a new social network “friend” on the other side of the world, whom we barely know and often forget in a matter of a few hours or days.

Yet all these interactions have become fodder for a new industry that secretly vacuums up the data and preserves it forever on high-end servers that hold many petabytes (a million gigabytes) of information. This industry offers new tools to search that data and reconstruct our past, and even our real-time movements via our mobile phones, in a way that could well come back to haunt us.

WikiLeaks has just released the Spy Files – a trove of almost 300 documents from these companies that shine a light into this industry. At the Bureau of Investigative Journalism, where I work, we trawled through these documents, and tracked down yet more material which our research team – Matthew Wrigley, David Pegg, Christian Jensen and Jamie Thunder – used to create an online database that will soon cover over 160 companies in some 25 countries.