To Improve Data Quality, Sometimes the Best Place to Start is at the Very End

Brad Shimmin – Research Director, Business Technology and Software

Summary Bullets:

• With big data and analytics, older ideas like predictive analytics and AI are coming together to solve long-standing problems, most notably data quality.

• Sisense is adding another twist by taking advanced design and visualization concepts and putting those to work at the very beginning of the analytics lifecycle.

Invention invariably involves theft. Each generation of inventors stands on the shoulders of its predecessors, borrowing freely from their available pool of knowledge. Ideas are deconstructed, mixed up, and reapplied in new ways and within unexpected contexts to form, well, something new. Sometimes these new inventions are simply the opportunistic reinterpretation of an existing idea, taking something unique but impractical and turning it into something incredibly useful. That’s the way it was with the invention of the automobile, the light bulb and the radio. And that’s how it is with big data and analytics, where older ideas are only now coming together to solve long-standing problems.

That’s how it was with the sudden availability of machine learning (ML) techniques. With the creation of accessible developer tools and the affordability of AI-centric hardware, anyone can (to a degree) use ideas that were impractical 30 years ago to easily predict the future today. Now we have another recontextualization emerging within BI and data discovery solutions. Already vendors like Qlik, Tableau, IBM, Microsoft and many others have put ML to work, blending it with predictive analytics to empower a much broader array of data analytics and business users. For example, ML-infused software can predict which data source will best answer a question or which chart will best represent that answer.

One of the benefits of blending AI and predictive analytics in this way is that it can be used to greatly improve data quality. This improvement is greatly needed within any analytics-driven organization where the proliferation of personal, public, cloud and premises data has made it nearly impossible for IT to keep up with user demand. And that brings us to Sisense, a comparative newcomer to the “modern BI” movement. With its recently introduced 7.0 update, the vendor introduced a new idea, termed dynamic data mapping.

Basically, Sisense wants to improve data quality by taking advanced design and visualization concepts typically reserved for the final product of a BI solution, namely dashboards and reports, and put those to work at the very beginning of the analytics lifecycle. Instead of simply making sure a data set is clean and appropriate to the task at hand (using AI, let’s say), SIsense users can also apply data visualization tools themselves to the task of data preparation. The vendor suggests that users can visually zoom into a given data set and then zoom out to see it in a broader, holistic context.

Is this a revolutionary new invention? Of course it’s not. As a matter of fact, Sisense’s rivals (likely led by Qlik, which has invested heavily in this sort of effort) are likely to position dynamic mapping as just a flashy wrapper around a logical evolution of data preparation. But is it inventive to simply reapply an older idea in a brand new context? Does it make data preparation a much more practical endeavor? Yes and absolutely yes.

Whether it’s the application of AI algorithms or the reapplication of basic data visualization tools in the service of data preparation, if the outcome is improved business insights shared among more people, then I think it’s safe to say that whatever the means, the outcome is innovative. Sometimes it is best to start with the end in mind.

What do you think?

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.