I had a funny conversation last weekend with a consulting peer that mentioned governance with self-service BI tools. I don’t think that person realized this was a HOT button topic with me. Many companies struggle with this topic, numerous Bring Your Own Reporting Tool solutions being used/introduced, how to best combine these solutions with their enterprise BI platforms and with business users driving these implementations…they do need a little help to avoid common pitfalls. I started writing my self-service BI governance book last year and I am still slowly plugging away at it. I will have a complimentary webinar on this Self-Service BI Governance topic on February 28, 12:00PM EST.
I have been preaching governance with self-service BI tools since at least 2010 when Excel Power Pivot was released. IT and DBAs did not want to allow it and feared it. Unless Microsoft taught customers how Excel Power Pivot could be managed and controlled, it would never have been embraced like it is today. In 2012 in Atlanta I hosted a self-service BI governance workshop for enterprise accounts since literally everyone I talked to was struggling with this topic – including some the biggest enterprise firms in the world. Let me tell you, that workshop was standing room only with BI directors, data warehouse managers, business unit management, etc. They came and they stayed for the entire day. My point is that FEW groups have figured out a good self-service BI governance plan and with all the new, slickery, BI tools that are flooding their businesses. Many groups are having a tough time balancing control and business empowerment. It was refreshing to hear highly esteemed, industry leaders openly admit their pains, lessons learned and share best practices. Peers helping peers – it was a fantastic event.
Large companies do suffer with shadow IT systems, rogue Access databases with linked tables and enterprise database quality servers hidden under analysts desks, true Excel-hell and now a plethora of different BI tools and devices. Enterprise self-service BI deployments do require governance structure, processes and workflows – it truly is a critical success factor. If you simply install the BI tool and start rolling it out enterprise-wide with no governance in place, you WILL have an uncontrollable mess. I have seen these messes and cringe a bit when I deliver the diagnosis for fear of shoot the messenger. These kinds of messes can be expensive to fix. They also can introduce a variety of reporting risks, different definitions of measures, duplication of data that is not all updated/is in different states by data source, a mess of “blends/mash ups” that make it harder and harder to create reports as the system grows, when users don’t understand things like slowly changing dimensions in year over year reporting there is high risk that incorrect business reporting structures and values for history are being reported on, etc. It is sooooo much better to roll out self-service BI the right way at the start. Resist temptation, impatient users and have a governance plan in place…then roll out the fabulous, modern BI solution.
The Microsoft Excel Power Pivot engineering team knew the importance of self-service BI governance and built Power Pivot Management Dashboard in SharePoint in v 1.0. They took it further with the SharePoint Audit and Compliance Server and the Excel 2013 Inquire. I have several video demos of these features on my YouTube page. With Excel Power BI, they are again building in governance for Office 365 Power BI Sites, Shared Data Sets and Power Query right in the v 1.0 offerings. Tableau has had monitored data source access, usage auditing, logging, performance reporting and customized compliance reports. QlikView that I often ding has had a governance dashboard for a few years now. BI governance features are built into many BI tools, both self-service and traditional BI platform offerings. What is not “out-of-the-box” per se, is the customized people, process and environment part of the equation that varies from client to client.
Looking back over 10 years of my asset archives now stored in the cloud – I was too lazy to pull floppy disks and Iomega Zip drives past 10 years. I found old material on governance for this exact topic dating back to 2005. Governance with self-service BI is NOT at all a new issue. Consulting firms have been helping companies for years roll out ad-hoc reporting systems with and without data warehouses and data marts. What is NEW? The high quality, quantity and accessibility of self-service BI tools in the market today. The extreme pace of business in a global 24×7 connected world. The massive, constantly, exponentially growing data volumes. There are many more data source types than ever before and today data is more widely dispersed across hosting providers, data centers, cloud providers and on-premise data centers. Business users today are far more savvy than 10 or 15 years ago. They are also more demanding with higher expectations…rightfully so since modern BI technology is amazingly awesome. BI and IT developers these days are often perceived as overhead versus truly strategic resources. The big picture game has not really changed much at all. The same Decision Support/Information Delivery design patterns used a long time ago still do apply today with some tweaks for what is new.
When I was at Capgemini, we called self-service BI governance Information Delivery assessments, blueprints or reporting governance engagements. These Information Delivery projects were different from Master Data Management and Data Governance engagements that were done at the deeper entity, data attribute and value level. These fun, strategic Information Delivery engagements were customized for a client’s goals, environment, MDM system if any, Data Warehouse and Marts, Source Systems, User Personas, BI and Reporting Tools, Compliance, Regulations and so on. You don’t need to engage a Big-4 firm and pay $200,000+ for this kind of engagement. This might be something your team simply does with a few guidelines or some mentoring. For larger groups, it may help to engage a seasoned consultant (wink) to provide proven tools and templates. In the meantime, I will share a high level, partial outline with the types of topics you should be thinking through. There is quite a bit more to it but these little snippets should provide you a good running start.
High Level BI and Information Delivery Strategy Topics
- Areas of the company and overall BI where the Self-Service BI tool will be integrated/used
- The mix of available self-service BI tools, why and when it makes sense to use them
- Personal, Team and Enterprise ETL and data cleansing
- Personal, Team and Enterprise Reporting and Dashboards
- Onboarding, Communication, Training and Support
- Deploying Self-Service BI Server within the Enterprise
- Integration with exiting portals, applications and business processes
- Self-Service BI Server Technical Architecture
- APIs, and Utilities
- Self-Service BI Server Implementation Best Practices
- Roles and responsibilities
- Define the types of users, skill levels, types of access
- Security Access, Authentication, Authorization
- Creating Reporting Sites, Groups and Projects
- Define centralized, shared data access management and monitoring
- Data Source Level Security
- Data Level Security
- Data Loads and Refreshes
- Management Permissions
- Distributed Environments
- High Availability
- Back Up and Recovery
- Automated Services and Processes
Development and Deployment
- Semantic Model / Shared Data Sources or Shared Data Sets
- Metadata capture and delivery
- Process for combining Semantic Models into a Single Enterprise View
- Dashboards and Report Version Control/Change Control
- Developing, Testing , Reviewing, Approving, Publishing and Distribution
- Data Quality, Issue Management and Triage Support
A Phased Self-Service BI Roll Out Roadmap