When I started to read about the concept of Big Data, I was at first a little skeptical about the drivers, so I thought I would do some research. Wikipedia definition:
Big data are datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage, search, sharing, analytics, and visualizing. This trend continues because of the benefits of working with larger and larger datasets allowing analysts to “spot business trends, prevent diseases, combat crime.” Though a moving target, current limits are on the order of terabytes, exabytes and zettabytes of data. Scientists regularly encounter this problem in meteorology, genomics , connectomics, complex physics simulations , biological and environmental research , Internet search, finance and business informatics. Data sets also grow in size because they are increasingly being gathered by ubiquitous information-sensing mobile devices, aerial sensory technologies (remote sensing) “software logs, cameras, microphones, RFID readers, wireless sensor networks and so on.”
What I found is kind of strange, a lot of people talking about it sayings its going to be a big thing, and I guess that makes sense, but these may be the same people that told me the computer was going to free up so much of my time after 1990 I would have most of my day to relax…that didn’t happen.
I see so much hysteria around these kinds of topics that I often wonder if its solely a marketing technique. I think there will be some arenas where this is an issue in the near future, potentially E-commerce and Smart Grid companies may face some of these challenges, but I am also confident those industries will find solutions because vendors are good at finding solutions given enough time.
As an FYI, Smart Grid data and E-commerce processing happens all through the cycle, not just in the storage environment, so robust real time analytics may be a really good business idea.
About the Author
Dave Tyson– MBA, CPP, CISSP
As President and CSO of Apollo, Dave Tyson leads CISO Insights, the cybersecurity advisory and professional services business unit of Apollo Information Systems. Dave partners with Apollo’s clients to provide …Read More