Traditionally BI systems are geared to answering questions from users and to report on deviances from pre-setup criteria. This means translates into long costly implementation projects as well as regularly being told it was not designed to do that. We are currently developing a solution to address some of the typical problems that user of BI solutions experience.
- Partial Fit
- The 80/20 principle applies to BI projects, much as any another IT solution. Most of the requirements, in terms of content and accuracy, can usually be achieved within time and budget. The last 20% however, is responsible for the failure of many BI solutions. The difference between success and failure here is fairly simple. The solution succeeds if it's able to deliver 8 out of 10 reports perfectly and the other 2 only partially or not at all. It fails if it delivers all 10 partially. In other words, it's better to deliver fewer reports, as long as they're perfect, than to deliver all reports imperfectly.
- This is because it is often impossible to complete a report if the underlying data model doesn't expose all the required information. If you have to supplement information, missing from data warehouse, to deliver a report, its pretty much game over. The cost, performance, supportability and security implications almost negates any benefit derived from the data warehouse.
- Exceptions
- How often do we say "yes, that's the number in our systems but we apply this rule in our reports". When business logic or information isn't held in the source systems, but rather in informal or unstructured locations such as spreadsheets, it has to be added into the data warehouse, because adding it to each individual report is counterproductive, error prone and often complex. Adding this information to the data warehouse is often complicated, expensive, and it may even require new user interfaces or front-end applications to allow maintenance and administration.
- Manual Operations
- Hard Coding
- Overrides
- Inflexibility
- Once a data warehouse solution goes live, it is typically very difficult to change or extend the data model. Traditional data warehouse designs require schema changes to facilitate new content because their structure literally represents the business model. This could be as simple as adding a column, but in worst cases, it requires the restructuring of existing tables, with severe consequences for existing reports.
- While too little flexibility results in slow delivery and poor adoption, too much flexibility can cause chaos and confusion. It is therefore critical to develop a solid version control process for all changes to a data warehouse, regardless how easy the changes may be.
- Data Integrity
- The accuracy of the data warehouse depends on the quality of data in the source system. An incorrect date of birth may not be a big issue, but an invalid one almost certainly is. By nature, a data warehouse tries to expose trends, patterns and outliers. If the source system fails to validate an entry, resulting in a 300 year old customer, it could significantly skew the BI results. Unfortunately it is very difficult to anticipate the degree of validation and cleansing required without a comprehensive review.
- Cost
- BI solutions tend to be prohibitively expensive due to the proprietary software required. We are using Open Source where applicable to bring the cost down. Some of the unexpected benefits of this has been performance and flexibility.
|
|