The Petabyte Age

Both the ways and the amounts of info-collecting about you, about each person on the planet, about all life, has likely gone far beyond the ability of most of us to comprehend. The shrugging off of the recently passed new law on immunity for previously illegal eavesdropping is surely proof of that. It isn't about protecting you - it's about preserving the controls of power, and insuring that all the information available is collected, tagged, organized and stored for uses both known and unknown.

It's changing the game, not just the rules.

The vast majority of people imagine that spying on someone, wiretaps we so quaintly call it, involves some movie-style human observation of another human. The tech today is light years away from such action. That's why the government moved so fiercely to include electronic surveillance and change the FISA law.

Machines collect data at levels most of us - me included - have trouble grasping. Wired magazine's July issue does a good job of explaining some of this, that we are already in the Petabyte Age. What's a petabyte? Well, consider that 1 terabyte is about a $200 hard drive capable of holding 280,000 songs. There are 1,204 terabytes in 1 petabyte - currently Google can process 1 petabyte every 78 minutes. It's a number that will soon change too.

"The Petabyte Age is different because more is different. Kilobytes were stored on floppy disks. Megabytes were stored on hard disks. Terabytes were stored in disk arrays. Petabytes are stored in the cloud. As we moved along that progression, we went from the folder analogy to the file cabinet analogy to the library analogy to — well, at petabytes we ran out of organizational analogies."

Wired examines how in just the last few years, data collection of enormous magnitudes are changing the way we live now and will live soon - dealing with, among other things, astronomy, biology, news-tracking, political databases. The full list of their recent report is here.

So most of us just can't conceive of how the new laws regarding FISA have removed most any kind of personal privacy, or that it apparently arrived by telecoms purchasing the influence needed to pass the law.

"Maplight.org analyzed the contributions to both sets of the Democrats and found that those who switched their votes received, on average, 40 percent more money in campaign contributions over the last three years from Sprint, Verizon and AT&T's political action committees.

On average, those who changed their votes collected $8,359 dollars from those PACs from January 2005 through March 2008, while those who did not change their opposition collected $4,987.

For all House members, including Republicans, those supporting immunity collected nearly twice as much money from those PACs than those who did not: $9,659 to $4,810.

Maplight.org was careful not to say that any member's vote was purchased, but says the correlation raises questions.

Correlations -- that's what the data collection is all about:

"There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot."