It would be fantastic if there were only one marvelous multipurpose database to fit each and every job, one super application that was the best at transactions, operations, warehousing, reporting, and analytics, with beautiful eye candy, reality paints a different picture. But in the real world, real users know it comes down to having the right database for the right job.

The original big players in the database space, namely DB2 and Oracle, can be thought of as the transaction leaders. They have proven time and again that they can manage OLTP environments in various industries. Then there was the warehouse innovation. Store the current state of the date in another database for reporting. Excellent, a new source of revenue for the same database technology. Other vendors realized that these databases were not optimized for reporting and so new vendors entered the market. As well, front end interfaces, which provided fancy graphical capabilities, had to integrate with these environments. Unfortunately the amount of effort to get acceptable performance resulted in adding summary tables, denormalized views, temporary data stores and other transformations. Along came cubes, an approach to deliver high performance answers to the graphical user interfaces. The results are excellent for the questions that we already know about. Umm, but I need to also know about these other attributes? No problem, we’ll just create another cube with those attributes. And so on , and so on, and so on. Can anyone see the problem?

This constant attempt to retrofit existing technology to meet the needs of all stakeholders has led to

– Tying existing databases to hardware platforms (appliances), which seems to be a great solution for the hardware vendors
– In-memory solutions, which beg the question — if I continuously have to add more disk space, how can in-memory solve my issues since it costs more and is does not handle as much volume?
– Reporting tools with caching, which are just another cube or in-memory solution but moved over to the application server or client side, causing constraints on more touch points.

The bottom line, as we’ve said before, is that not all data is created equal. Data moves through a lifecycle and its value to the enterprise, and the best tool for extracting that value, moves with it.

I can see row based databases as the transaction stores for operations, unstructured or semi-structured data that requires search style access, a column store for the analytical warehouse or mart to drive business intelligence, highly compressed data stores for storing logs, and a front end that provides the reporting capabilities.

In other words, the right tool for the right job, at the right time.