Big Data. It's big, but is it clever?

Presentation from Autonomy
Fundamental shift in the "I" in IT. Now all about information. Most of it doesn't fit into neat databases. Eg social media, rich media, clickstreams, sensor data, images, email and more.

All unstructured. Growing rapidly. It's where all the interesting things happen. But, how can we analyse it? Human information contains emotions, nuances etc. Difficult to derive meaning.

Processing human information very complex. Traditional searching is based on attributes. Not looking at concepts and meaning of the information.
Human information is diverse, many different types. Ideas have the concept of distance, no two are the same. Can use very different words, but the concept can be the same. Context is also important. Your point of view can change the context of the information.

Meaning based computing is the answer! Moves away from keywords to ideas and concepts. Has to be done in real time. What's happening not what's happened.

An example. Analyse calls to a call centre to find out what customers think of customer service. Sentiment analysis on all contacts, email, calls etc. using intelligent queries. Apparently it works, and has been used in many areas.

We all have a lot of valuable information locked up as unstructured data, which we can now analyse. Social media, twitter, blogs etc can all be used to extract information.

So, should we be using this sort of analysis, and if so, what for?

- Posted using BlogPress from my iPad