Thoughts on Column-oriented Database Management Systems
As a new contributor to the SAND blog, a few words of introduction are probably in order. I have been working at SAND Technology for 12 years (already!), predominantly in research & development and implementation of Data Warehouses and Intelligent Information Management systems, and I am currently Vice-President of R&D and Deployment for the company. My background is in mainframe OS system programming, mainly Assembler, and APL (my specialization at university) – a peculiar mix which should give you an idea of my age.
Since the beginning of 2008, I have been trying to reduce my use of the car. So, instead of driving in to work, I’ve been using public transport in the form of a pretty comfortable bus. This has increased my commute time by 30 minutes, but now I have a good two hours in which I can listen to music, read technical documents, do development work and management planning – and write blog posts! At the end of the day, I find this is better way to spend my time than behind the wheel of a car in traffic: not only am I reducing my negative impact on the environment, but there is less stress, more productive work gets done, and overall I have more “quality time”.
One of the ways I have used my newly available time was to write a white paper for SAND entitled Corporate Information Memory. In my blog posts, I will be going deeper into some of the ideas explored in that paper, and will also share my point of view on a variety of related technical subjects. I welcome any feedback or suggestions as to topics you would like to see discussed.
From a technological standpoint, I am a strong believer in Column-oriented Database Management Systems (CDBMS), also referred to as columnar databases or CBAT. I am very pleased that more and more companies have accepted CDBMS as the only way to go in addressing the challenges presented by very large data warehouses. At SAND, we have known this for over 10 years, and it’s great to see more companies and media talking about our underlying strategy. Just five years ago, a column-based database was considered exotic. Now this is no longer the case, and we don’t get the “that’s too good to be true” reaction when we describe what our products can do.
An increasing number of companies, having spent large amounts on databases that use traditional row-based architectures, are starting to look at different approaches. This is mainly a response to the need to keep more and more data accessible, coupled with new regulations calling for retention of more data for longer periods – leading to what has been poetically described as a “data tsunami” or “data storm”). Companies looking for solutions to reduce their TCO and improve their ability to meet Service Level Agreements are discovering column-based architectures, selective data encryption, database compression and new concepts of Intelligent Information Management (at the moment, mainly consisting of Information Lifecycle Management principles). This is great news for us at SAND, because these are precisely the areas in which we excel!
In the next post from my seat on the bus, I will discuss some of the technical advantages of CDBMS.