Big Data

IT Management / September 4, 2011 / by Dave Tyson

When I started to read about the concept of Big Data, I was at first a little skeptical about the drivers, so I thought I would do some research. Wikipedia definition:

Big data[1] are datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage,[2] search, sharing, analytics,[3] and visualizing. This trend continues because of the benefits of working with larger and larger datasets allowing analysts to “spot business trends, prevent diseases, combat crime.”[4] Though a moving target, current limits are on the order of terabytes, exabytes and zettabytes of data.[5] Scientists regularly encounter this problem in meteorology, genomics[6] , connectomics, complex physics simulations [7], biological and environmental research [8], Internet search, finance and business informatics. Data sets also grow in size because they are increasingly being gathered by ubiquitous information-sensing mobile devices, aerial sensory technologies (remote sensing) “software logs, cameras, microphones, RFID readers, wireless sensor networks and so on.”[9][10]

What I found is kind of strange, a lot of people talking about it sayings its going to be a big thing, and I guess that makes sense, but these may be the same people that told me the computer was going to free up so much of my time after 1990 I would have most of my day to relax…that didn’t happen.

I see so much hysteria around these kinds of topics that I often wonder if its solely a marketing technique. I think there will be some arenas where this is an issue in the near future, potentially E-commerce and Smart Grid companies may face some of these challenges, but I am also confident those industries will find solutions because vendors are good at finding solutions given enough time.

As an FYI, Smart Grid data and E-commerce processing happens all through the cycle, not just in the storage environment, so robust real time analytics may be a really good business idea.


Scroll to Top