Categories
Internet of Things Research & Development

The Big Data Bandwagon

Big Data and I go way back. How can I get on the Big Data Bandwagon?

It's not a domain into which I regularly stray but nuclear physics was the focus of both my parents' careers so their use of computers comes to mind whenever the topic of Big Data comes up. I'm stepping out of my comfort zone but I am going to hypothesize that the study of physics and the Internet of Things share certain attributes and that both are sexy because they are part of Big Data. You're looking at Big Data in the illustration here.

Both physics and IoT begin with the assumption that there's more to the world than what the naked eye can see or can be detected by any of our other human senses. You can't see atoms, or electrons or quarks or any of those smaller particles. All you can use to know they are there are the measurements of their impacts on other particles using sensors.

And sensors are also at the heart of the Internet of Things. In addition to the human-detectable phenomena, sensors embedded where we can't see them detect attributes we can't see, don't have a smell, don't make a sound or otherwise are too small, too large, too fast or too far away for us to use our "native" human sensors to detect. The sensors monitoring the properties of materials in physics (like the sensors in our environment monitoring the air quality, the temperature, the number of cars passing over a pressure sensor on the roadbed) communicate their readings with time stamps and these contribute to other readings as a set of data forms.

You get the rest: the raw data then become the building material upon which analyses can be performed. It's difficult for the common man to discern patterns from the illustration above or millions of sensor readings from a nuclear power plant. Machine learning and algorithms extract the patterns from the data for us and we use these patterns to gain insights and make decisions.

So, my point is that the concept of using computers to analyze large data sets to answer all kinds of questions–the core of Big Data–has been around the research community for decades and applies to many, if not all, fields. IBM has long been leading the charge on this. Here's an interesting project led by Jeff Jonas, Chief Scientist of IBM's Entity Analytics Group, that just celebrated its one year anniversary. A January 2012 HorizonWatching Trend Report presentation on Big Data points to lots of resources.

What's new with Big Data in 2012 is the relative ease with which these very large data sets can be reliably collected, communicated, stored and processed, and, in some cases, visualized.

A feature article about Big Data's relevance in our lives in the New York Times frames the subject well and then explains why Big Data is trending: everyone wants to see the past and the present, and to understand the world more clearly. With our improved "visibility" we might be able to make better decisions. The "text book" example is the Oakland Athletics baseball team comeback on which the book and movie, Moneyball, are based.

With the help of coverage in books, motion picture, major news media and tech bloggers, Big Data is one of the big memes of 2012. Trends like the widespread adoption of Big Data usually lead to large financial gains.

Let's see if I can use this data to make better decisions! Maybe I should re-brand everything I do so that the relationships of my activities to Big Data are more clear to others. Big Spimes? What do you think?

Leave a Reply

Your email address will not be published. Required fields are marked *