Trends Driving the Future of Analytics

Posted by  Jen Underwood   in  , , , ,      8 months ago     11097 Views     2 Comments  


Another sleepless night in San Jose, California without my beloved Diet Mountain Dew until tomorrow morning… Seems like a perfect time to share a few thoughts on awesome future analytics industry trends that I am monitoring. As I continue to grow and stretch domain knowledge across numerous vendors, I am also being pulled more and more towards futures analysis.

Reviewing futuristic innovations as an engineer at heart is incredibly fun, interesting and also requires expert alerting, filtering and skimming through massive industry, social and media noise to find golden nuggets. When you do find a fascinating innovation in infancy, the commercial applications for it may not be obvious. This is where you need to be creative visually yet remain totally practical “guestimating” what abstract concepts may become viable, commercial applications and what might fail. Two of my favorite “tech futures” thought leaders so far are Daniel Burrus and Ray Wang.

Here are few future technology trends that I feel will significantly influence analytics application designs soon if they have not already begun to do so.

– Embedded Analytics (already prevalent today and simply growing)
– Cognitive BI (the IBM-Watson effect, Siri, Natural Query, etc.)
– Automation of BI and Analytics
– Hyper-Individual Experiences
– Marketplaces
– Cloud (also widespread today and growing rapidly)

Let’s take a peek at a few of these trends starting with Cognitive BI and Automation. In future articles I will dig into a few others.

Cognitive BI and Automation

Cognitive systems in general will transform how organizations think, act, and operate in the future. “Smart machine models” will be designed to analyze unstructured data, video, images and human language via artificial intelligence and machine learning algorithms. A few players in this space include IBM, Data RPM, AlchemyAPI, Digital Reasoning, Highspot, Lumiata, Narrative Science, Cortica, Ersatz, Semantria, Numenta and nViso. GigaOM does a lovely job covering bleeding edge vendors like these.

The feature image above showcases Digital Reasoning’s Synthesis cognitive systems technical solution architecture design that is already being implemented by early adopters. You can see big data, advanced analytics, artificial intelligence and even a little Tableau sprinkled in. As a Tableau fan, I’d totally love to hear/see a joint Digital Reasoning and Tableau session – hint hint if Ellie, Thierry, Sean or anyone over there reads my blog.

We are already seeing cognitive technology in the BI world with Watson Discovery Advisor, Analytics (Project Neo) and Explorer, Oracle Endeca, Data RPM, Targit, Power BI Q&A and other analytics systems using natural language queries to automate information discovery and reporting. What this means…users on big data systems will no longer be limited to querying information stored in predefined views, tagged semantic models or static data models. Data is being accessed via automated indexing – think Google search for BI apps/reports.


IBM Watson forever changed the technology world and how data can be used to uncover valuable insights. IBM’s Watson is an artificially intelligent computer system capable of answering questions posed in natural human language. Extensive research with regards to text mining and human language context was led by David Ferrucci and a joint team of IBM and major university research groups. The project was inspired in a restaurant where the show Jeopardy was playing the background. A few years later, Watson was featured on Jeopardy and won against the top players in the world. I will never forget that special day back in 2011 or the amazing behind-the-scenes television program IBM made sharing their Watson research journey, failures and successes. To build Watson, IBM used virtual teams and virtual reality meetings to get the best talent on the planet involved in that project.

Watson is significant because its advanced natural language processing can understand the complexities of unstructured data, that account for ~80 percent of the data in the world today – without tagging or modeling. It uses data science techniques to automatically generate hypothesis and evaluate a panel of responses based on relevant evidence. Essentially Watson is employing dynamic learning techniques, a little like neural network per se, that allows it to continue getting smarter based on outcomes with each iteration and interaction. Looking ahead, IBM is enhancing it to include foresight, planning, perception and extrapolation. That is truly mind exploding with regards to technological advancement. If this fascinates you, take a look at the IBM paper called The Era of Cognitive Systems: An Inside Look at IBM Watson and How it Works.


Another really cool aspect of is this particular project is the Watson Question and Answer API (QAAPI) for developers. App or BI developers can integrate Watson capabilities, as a service, through the use of a REST API. IBM is not at all the only player with this technology but I feel that they are the market leader.

Taking Cognitive BI one step further, a company called nViso is transforming facial imaging, human non-verbal signals and emotional responses into insight. This nifty innovation will be used for measuring customer satisfaction and a plethora of other applications. Oh boy, the fun we are going to have making silly faces at cameras in the real world to toy with this one, confuse marketers or express our dissatisfaction more colorfully than ever before!


More Automation

A former Microsoft peer of mine recently left the BI world for fear of automation. At first I thought he was overly paranoid but now I am beginning to see what he sees. There are increasingly more aspects of analytics applications being automated or simplified with technologies like Watson and Data RPM. I also saw something from esteemed Industry Analyst Neil Raden on Twitter about Microstrategy and automation – dying to know what Microstrategy is doing in this area.

As technology continues to improve and shifts to the cloud, the BI and analytics world will continue to evolve. There are already less data model building jobs and more what does the data actually mean jobs today. I forecast that trend to continue. I don’t know where ETL stands down the road. I am seeing ETL automation tools just like predictive automation and reporting automation with Q&A but I do remain a bit skeptical. Viable automation of BI does feel to be a few years out from becoming a reality. However, I am getting less skeptical over time and wondering what will be next, curiously exploring what is happening in the wonderful world of technology.


Jen is a Microsoft Sr. Program Manager of Business Intelligence & Analytics. She works with external groups, customers, channel partners, MVPs, BI professionals and application developers to better connect the outside world to engineering. Previously she was a Microsoft Business Intelligence Sr. Technical Product Manager for offerings spanning across Microsoft SQL Server, Excel and SharePoint. She also held roles as an Enterprise Data Platform Specialist and a Business Intelligence Consultant for Big 4 Systems Integration firms. Throughout most of her ~20 year career she has been researching, designing and implementing data warehousing, business intelligence and predictive analytics solutions across a variety of open source, niche and enterprise vendor landscapes including Microsoft, Oracle, IBM, and SAP. Jen is quite active in the global technical community as a presenter, author, blogger and volunteer. Jen was previously honored with a Boulder BI Brain Trust (BBBT) advisory membership, a 2013 Tableau Zen Master (MVP) award and a Dun & Bradstreet MVP. She writes articles for TechTarget's BeyeNetwork, SQL Server Pro magazine and other industry media channels. Jen holds a Bachelor of Business Administration degree from the University of Wisconsin, Milwaukee and a post graduate certificate in Computer Science - Data Mining from the University of California, San Diego.


  1.   January 16, 2015, 12:22 PM

    Great post Jen,

    are you planning on doing something with tableau + R (I see on your bio that you are interested in tableau as well). The integration between both is pretty simple (I played around for a while and documented my simple test here: if you want to see) but I think the community is lacking more advanced examples.


    •   January 16, 2015, 12:26 PM

      Hi Jen please ignore and delete this comment, I just found a post about it on your blog :)

  2.   January 15, 2015, 10:51 AM

    Thanks for the post, I thought I’d jump in to add some detail.

    Shiny is an open source R package that provides an elegant and powerful web framework for building web applications using R. Shiny helps you turn your analyses into interactive web applications without requiring HTML, CSS, or JavaScript knowledge. While the Shiny package itself includes a basic web server, it’s only designed to serve one application at a time. (

    If you want to put your Shiny application on the web, you need Shiny Server. Shiny Server is designed to serve up multiple applications on the same server. Shiny Server provides a platform on which you can host multiple Shiny applications on a single server, each with their own URL or port. It enables you to support non-websocket-enabled browsers like Internet Explorer 8 and 9, and is available under an AGPLv3 license or an RStudio license. Folks are provided pre-compiled binaries for Linux Only – Ubuntu 12.04 (or later) and RedHat/CentOS 5 and 6. Enterprise/Professional data science teams choose Shiny Server Pro over the open source version to secure user access, tune application performance, monitor resource utilization and get the direct support they need to create the best interactive data experiences for their customers and colleagues. Which version you use depends on your requirements. (

    We realized that some folks can’t stand up a Linux server so we decided to do it for them with – currently in beta. Now people can deploy their Shiny applications on the web (hosted by RStudio) in minutes in an easy to use, scalable and secure way. The free tier is currently limited to 50 active hours per month and 10 apps. We’d love to get feedback on the pricing model since we haven’t launched yet. (


    •   January 15, 2015, 11:58 AM

      Thanks for the fantastic detail Bill. I have been playing with the – currently in beta.

Leave a Reply

You can use these tags:   <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>