Data Glut: it isn’t only email…

Posted on October 7, 2010 · Posted in Uncategorized

I was reading an article about hi-tech airships in IEEE Spectrum when my eye caught in the sidebar a link to another article titled  The UAV Data Glut. What do you know – we thought Infoglut was a human problem, and now Unmanned Aerial Vehicles bitch about it too?

Naahh… of course, it isn’t the UAVs that complain; it is humans, the only species that can. The problem, according to the article, is that the super sophisticated drone planes generate more data than humans can look at: “In 2009 alone, the U.S. Air Force shot 24 years’ worth of video over Iraq and Afghanistan using spy drones. The trouble is, there aren’t enough human eyes to watch it all.” And it’s getting worse: the next model of Reaper drone will record 10 video feeds at once!

UAV

Image: Wikimedia Commons

So it isn’t only email that’s spiraling beyond our ability to read it all; potentially important military information is also doing it. And the analogy doesn’t end there: the two widely different cases both involve nuggets of value – an important email, or the signs of a camouflaged enemy vehicle – buried in tons of irrelevant data. In both cases, technology is enabling the arrival of ever more data without a commensurate growth in human ability to absorb it all…

The solution, the Spectrum article shares, will have to be relegating the analysis to computers (who else?…)  There are efforts underway to develop software that can watch the UAV’s boring data streams and identify those few needles in the haystack. Which makes you think – what about us knowledge workers and our flooded Inboxes? Sure, there are some tools that can help – Google Priority Inbox is one of the latest additions – but I suspect the military will get more powerful stuff going, and we’ll have to patiently wait for the civilian market spinoffs that always come later.

Who knows, maybe in a decade we’ll have software so smart that it will be able to write our outgoing mails, read the incoming mail, screen the junk and file the rest – while we take off to read a good book? 🙂