Business Intelligence in the Modern Era

This post offers an updated definition for BI, and suggests that you don't have to think about it as a box on an org chart.

BI has changed a lot in the last two decades. Technologies and best practices have evolved, and we've found more ways in which a BI program can deliver value. Some of these innovations have occurred outside of IT or the BI Competency Centers that many businesses have established. At the same time, many organizations are moving to make business units autonomous.

These changes lead many people to ask what exactly is BI? Is it a box on the org chart? Does it include analytics that were never done by IT? How do data governance and master data management fit in?

Business Intelligence Defined

I define BI as follows:

Business Intelligence:

The use of information to improve business performance

- Chris Adamson

The first thing to note about this definition is that it does not address any specific technologies or methods. These aspects change over time, and they certainly influence what we may be able to achieve.  But the objective is always to provide business value.

Secondly, note that this definition is not beholden to the boundaries of a departmental structure. Regardless of who develops, supports or uses solutions, it's all considered BI.

Let's take a quick look at both these aspects.

BI Services and Activities

The reason we commit resources to BI programs is simple: we intend to use information to deliver some kind of business value.  The definition has been crafted to cover any activities that support this objective.  It can be used to describe a variety of activities that provide business value, both old and new.

Among the older activities it covers:

• Traditional reporting, OLAP and ad hoc functions
• Dashboards and scorecards
• Traditional data warehouses and/or data marts
• Data integration services

At the same time, some newer uses of information are covered:

• Business analytics and predictive analytic
• Master data management
• Data governance
• Virtualization and federation services

The definition also covers activities that some people think of as on "the other side of the fence" from BI:

• Transaction processing

That's intentional; transaction processing manufactures much of the "raw material" that BI programs attempt to leverage.  When we plan an operational solution, we should be thinking about these downstream uses.

BI and the Org Chart

While you may have a group responsible for BI program management, it is important to understand that the scope of BI reaches well beyond this group. The delivery of business benefit from information impacts the entire organization.

Some of the functional areas that participate in BI are:

• Business units  All of the value from BI happens within business areas that use information. This is where decisions are made and impacts are realized.  For many businesses, responsibility for development of BI solutions also lies in business areas.  This is particularly the case for analytics, but also increasingly for the traditional forms of BI.
• BI Competency Centers  Whether part of IT or external to it, many organizations have established a centralized resource for planning and overseeing the development of traditional forms of BI, such as data marts, dashboards or scorecards.  In some cases, these centers have become focused on providing advisory services to business units that create and manage their own solutions.
• Analytic Competency Centers  Business analytics often begins within business areas such as marketing or risk management.  Analytic competency centers are developed to help other areas of the business leverage information in a similar manner. Whether part of the BI competency center or distinct from it, this is also a core BI function.
• IT  At a minimum, IT has some responsibility for the technical infrastructure on top of which information systems are built -- networks, computers and the services that keep them up and running. IT may also have responsibility for some of the business applications and data management solutions.

Regardless of how your organization structure divvies up these responsibilities, BI is the sum total of these activities, and not the domain of a particular group or department. A business strategy to create value through information cuts across many departments.  It cannot be planned or executed in isolation.

The Future of BI

We're not far from an age where BI is not a separate part of our information architecture.  We're not there yet, but several trends have us on this path:

• Focus on the future value and re-use of data managed by operational applications
• Commitment to data governance
• Maturation of master data management solutions
• Technological advances in data management and information access

When we finally arrive at a unified information architecture, the definition of BI will still hold. We will be closer to delivering on its promise than ever before.

And, without a doubt, we will have come up with ways of using information to deliver value that have not even be thought of today.

Document Information Requirements Graphically With BDM Diagrams

BI teams often struggle to keep the business engaged, especially during requirements analysis. This post looks at a graphical technique for documenting information requirements -- one that business people will read and respond to.

Keeping the business engaged is one of the keys to a successful BI program. One technique I have found to be very helpful on this front is Laura Reeves's Business Dimensional Model (BDM).

The BDM is a technique for documenting information requirements. Before I explain the BDM, a few words on the requirements themselves.

Information Requirements

Before you can design a dimensional model, you need to capture the business requirements that it will support. The most successful projects capture business requirements by working directly with people in the business, often through interviews or requirements sessions.

In my book, I suggest that as you organize your information requirements by business function.  You then state them in simple form: as a group of metrics and their associated dimensionality.

For example, a set of interviews about the taking orders might boil down to a requirements statement such as:

• Order Information by order date, order line, salesperson, customer and product.

The metrics that comprise the group are then fully documented.  For example, "Order Information" is further supported with documentation of:

• Order dollars
• Order quantity
• Cost dollars
• Gross margin dollars
• Gross margin rate

Relevant hierarchies in the dimensions should also be specified. For example, "Product" might be described as:

• All Products à Category à Brand à Product 

Finally, the major dimensions are cross-referenced to the metric groups in a conformance matrix.

These information requirements then drive solution modeling. The next step is to develop a top level dimensional model, and then a detailed database design.

(For more on developing and documenting requirements, including a fully fleshed out example, see my book -- it's listed at the end of this post.)

Getting People to Read It

When it comes to information requirements, you must ensure that the business stakeholders review and respond. (Better still is to involve the business in the identification and documentation process.)

In the book A Manager's Guide to Data Warehousing, Laura Reeves provides a graphical technique that helps keep the business's attention. She calls it the "Business Dimensional Model (BDM)."

This technique integrates nicely with the approach I've outlined above.

Each group of metrics is depicted in a simple diagram, with the metric group in the center and the major dimensions arrayed around it in circles.

For example, the Order Information metric group above might be documented thusly:

Within each circle, the underlined text identifies a dimension. Beneath the dimension, the level of detail applicable in the metric group is listed.

Additional illustrations document the dimension hierarchies. For example, the product dimension from the picture above might be documented like this:

The most detailed level of the dimension is shaded darkly. The arrows indicate hierarchies, going from summarized to detailed. Elements that will drive Type 2 slow changes have a shadow. Separate symbols (not shown) are used for junk dimensions, other derived elements, and future attributes.

People Like Pictures

I've found that using BDM diagrams dramatically increases the participation of business stakeholders. People look at BDM diagrams, understand them, and react to them -- often with great enthusiasm. That's a powerful aid in refining and validating your requirements.

These diagrams are also easy to produce using the built in drawing tools that come with basic productivity software.  This means you can often get business stakeholders to participate in their creation. For example, the pictures above were created in Microsoft PowerPoint using basic shapes and Smart Shapes.

Lastly, the ability to produce these diagrams using basic productivity software means they are easy to incorporate in the best format for this kind of documentation: the presentation.  I find the presentation format is far more likely to be reviewed than a word processing document. (More on this topic in a future post.)

Further Reading

As I said back in 2009, I am a big fan of Laura Reeves's approach to requirements and design. As you can see, there is a natural affinity between the BDM and the techniques I've talked about in the past.  I encourage readers to check out her book (see below).

More info about requirements and documentation can be found on this blog. Have a look at these posts:

• I first mentioned Laura's book in this blog back this post from 2009:  Recommended Books on the Data Warehouse Lifecycle (July 27, 2009)
• For an explanation of the three levels of a dimensional model (Requirements, Top Level Design and Detailed Design) see Dimensional Modelers Do Not Focus on Logical vs. Physical (July 5, 2011)
• For an example of a conformance matrix, see The Conformance Matrix (June 5, 2012)
• For an explanation of the type 2 slow change, see For Slowly Changing Dimensions, Change is Relative (October 9, 2007)

You can read more about the process of identifying information requirements in these books:

• For a full explanation of the BDM, see  Laura Reeves's A Manager's Guide to Data Warehousing (Wiley, 2009).  The BDM is covered in Chapter 7, "Modeling The Data For Your Business"

• The examples in this post are drawn from my book, Star Schema: The Complete Reference (McGraw-Hill, 2010)  A more fleshed out explanation of tasks and deliverables, with examples, cab be found in  Chapter 18, "How To Design and Document a Dimensional Model."  The examples from this post come from Figure 18-4 (which in turn builds on the star in Figures 3-3, and the hierarchies in Figure 7-3).

You can help support this blog by using the links above to purchase these books from Amazon.com.

[Edited 2/13/14 - Corrected the links, thank you for the emails.]