Workgroup or departmental business intelligence solutions are gaining traction in the market. Many business units are taking control of their BI destiny by building custom reporting systems, often consolidating data from multiple source systems including the Enterprise Data Warehouse (EDW). This is further evidenced by the success of QlikTech’s QlikView product which has gained the most notoriety in this space and new entrants including Microsoft’s Gemini and LyzaSoft’s Lyza. Even traditional vendors are starting to make a play for this segment of the market, most notably MicroStrategy. Gartner believes “that despite the inherent risk in silo perpetuation, workgroup BI’s light-touch nature will prove attractive in a recession.”

Definition of Workgroup/Departmental BI: A reporting solution focused on a specific subject area serving a small community of business users whose interest mainly lay with the targeted subject area.

The Why

Business objectives drive data needs, however traditional EDW approach cant keep pace with evolving business objectives and the resultant data needs. This is especially true when involving new data sources or complex new metrics. The traditional approach has processes in place that serves the EDW well in maintaining a reliable, accurate, and clean data environment. Moreover, the IT teams maintain a more complex tool set which can be intimidating to the average excel junky.

Departmental BI systems often consist of dirtier data (even with manual cleansing), questionable reliability, incomplete context, and a higher propensity for errors. They can also be extremely time consuming to maintain when managed by a sole business analyst with minimal IT support. With all these flaws business units embrace these solutions for their ability to provide information rapidly and adapt quickly to changing requirements. By providing information closer to the “speed of thought,” they are fulfilling some aspects of the promise of BI that cannot be done with the traditional EDW.

A TDWI research report by Wayne Eckerson, points to other reasons of silo perpetuation including politics, business events such as mergers and acquisitions, and business reorganization. However, the main reason they site is how companies empower business units with “budgetary authority” to “adapt products and services to local markets.” If business units thought they could perform better by integrating data with the EDW, would they squander their “budgetary authority” on building reporting systems from scratch?

Of Course Its Not That Simple

The problems with silo’s doesn’t change even though the technology enabling them has. As practitioners of data warehousing and business intelligence we’ve been drilled on the evils of these silos amongst which include: information babel, multiple versions of the truth, and the costs of supporting redundant bi teams, hardware, and software. The same TDWI report found that analytical silo’s can increase data warehousing costs by 30-50%, especially when implemented as a physically distinct system. According a survey they conducted the average organization has 2 data warehouses, 6 datamarts, 4.5 ODSs, and 28.5 spreadmarts.

Are the costs of redundant systems and staff worth it? It really depends on who you ask. Companies should focus on the ‘why’ rather than jumping head first into consolidation programs. The most successful solutions will find ways to bridge the gap between business needs to be flexible and fast with IT needs to maintain reliable standards driven environments.

Bridging the Gap: The Aellament Way

At Aellament we are targeting solutions to address these issues specifically. The basic premise of our strategy, as suggested by the title, is to use Departmental BI as a platform for Enterprise Data Warehouse Integration (EDW).

We believe that customers that standardized on the MicroStrategy platform or customers that are planning to standardize on the MicroStrategy BI platform are in a unique position to tackle silo perpetuation. With MicroStrategy 9, there now exists an easy pathway to migrating departmental BI into the enterprise BI stack. Two key developments make this possible: 1) the newly released Graphical Architect which significantly simplifies the most complex step of a MicroStrategy implementation, and 2) the release of a widely available free version of the platform which can address most BI needs of the average department or workgroup.

The key is to keep silo costs as low as possible while enabling business units to rapidly satisfy basic BI needs. Our 3 phased approach with minimal emphasis on ETL in the early stages perfectly balances the business needs of flexibility and speed with the EDW needs for stability and process driven programs. The following is a summary of each phase:

• Phase I - Setup and Deployment: Rapid deployment of a departmental project or the rapid conversion of a departmental project into a MicroStrategy project sitting on top of MySQL. This phase is to be completed with zero Transformations. Its basically a fork lifting of data from the source systems into the MySQL database (EL instead of ETL). In most cases departmental BI users are looking at a subset of corporate data over a shorter period of time which makes MySQL a good db option. Most of the work should be done by a BI expert while providing basic training to a department level power user. By the end of this phase several basic reports and metrics should be delivered to the business.
  

• Phase II - Parallel Development: The power user and the BI expert work together on creating additional reports and metrics. During this phase the BI specialist will provide more advanced training which include Architect, and Advanced Report Development. They will setup an ETL MicroStrategy project to apply simple transformations to the data. Additionally, in the second part of this phase some conformed dimensions from the EDW will be ported over to the MySQL db. However, the ETL work applied to integrate the data will be minimal. By the end of this phase the datamodel of the view layer will be further refined and very close to final datamodel that will be integrated into the EDW. Business gets more advanced reports and metrics with some conformed data; EDW gets a datamodel draft, testing on conformed dimensions integration, and a live requirements doc.

• Phase III - Integration and Migration: By the beginning of this phase most of the development on the departmental BI will be driven by the power user and other users within the department. The project will be setup with easy report builders so that end users and power users can be self sufficient. They can call on the BI specialist for advanced support. However, the main focus of the BI expert will be to integrate the departmental project into the EDW stack. Having provided training to at least one power user during the earlier phases frees up the time of the BI specialist. Here we bring on ETL development resources and step through the EDW processes for etl development and productionalization. The MicroStrategy projects from the prior stages will simply be duplicated and tweaked to run off the enterprise servers. At the end of the phase any hardware or software used by the department can be re-purposed for other departments.

Its quite difficult to capture all the nuances here as the process will have to be adapted for each environment. The common variables affecting implementation are the complexity of departmental BI programs, and software stack they are running on. With the graphical architect we believe we can rapidly convert most existing programs in a relatively short period of time. By using the free version of MicroStrategy we can get business units quickly started without huge capital expense upfront. At the department level BI costs mostly accrue from new software and hardware acquisitions rather then personnel. This is because they usually have a power user within the team that initiated the original program and does the basic work to maintain it.

By using this 3 phased approach time normally spent in documentation and iterative interviews is instead used to develop a live requirements document. Additionally, by allowing business to get their hands dirty with production reports we can significantly reduce user acceptance testing (UAT) time during EDW integration and the business can take immediate advantage of data driven opportunities. Moreover, in our experience adoption rates of BI tools are much higher when programs are initiated by the departments. The main reason is the sense of ownership they feel with tool as a department. Our approach initially gives control to the bussiness units and later becomes centrally managed. Bt the time users are on the enterprise stack they will be very comfortable with the tool. At the conclusion of the program a potential silo will have been eliminated.

For more information or a better understanding on how to adapt this process to your environment contact us at info@aellement.com.

ajo BI Industry, Strategy departmental bi, enterprise data warehousing, lyza, microsoft gemini, qlikview, workgroup bi