Sensor Data – The Ugly Duckling Of Big Data
May 2nd, 2013 by Jim Hayden
I thought I had seen it all when it came to big data challenges. Then I came to Savi Technology and got to dive deep into some sensor data and “ugly” doesn’t quite do it justice. While the sexy big data solutions around web, mobile and social continue to get tons of press and hype, sensor data continues to challenge organizations trying to handle the practicality of ingesting this information for simple event notification – never mind analytics and modeling.
We have already heard about the Internet of Things and know that the proliferation of sensors continues to instrument the planet. For those new to the space, sensors have become increasingly commoditized with active tags as low as $15 and passive tags at pennies so their use will continue to be pervasive. Even your smart phone is a sensor – with a large number of devices now GPS-enabled. Sensors can measure a variety of events such as location, temperature, shock and humidity to name a few and the data can come structured, unstructured or semi-structured. OTA and other firmware updates can change that output with no notice. It can come thousands of times per second to once per day and everywhere in between and more – and there is a LOT of it. If that wasn’t enough, the data is highly inconsistent – they run out of batteries leaving a huge data gap and are sometimes subject to harsh and abusive environments which can set off erroneous alerts such as a security/tamper event on a bumpy road. One very big, very unpredictable, very ugly duck.
But as the story goes, this ugly duckling is actually a beautiful swan. With our 23+ years of managing this data, Savi will help with that transformation, pioneering Sensor Based Analytics and enabling insights into this New Data. Today, we collect data from millions of sensors all over the world including our own as well as those of our partners. We are experts at separating the signal from the noise. And isn’t that what real insight is all about?
The value of knowing…