A wise mentor of mine used to comment on start-ups by saying “it is our job as executives to work out if there’s a pony in the room or just a room full of pony poop.” He would describe how, upon opening the door to a room full of poop, some people would slam the door and yell, “It’s a room full of poop!” Others, however, would think to themselves, “what’s making all that poop?” That second group of people will wade in and start digging around, and sometimes they will find something.

I look at Hadoop, I see all the interest and enthusiasm for the technology, and I don’t doubt there’s room full of poop here. And since Hadoop’s symbol is rightly an elephant, you can get a good sense of the scale of the poop I’m wading into here.

Now it’s important to remember that Hadoop isn’t a solution. It’s a framework. It lets you do some very clever things if you know how. That’s not uncommon. Looking at the last 30 years of software history, we can see a huge divide between the proprietary and the free and open source (FOSS) projects. And we see can see FOSS working out, even winning out in some places. If we look to commercial success (and I don’t mean “miners” who sell a company for great multiples, I mean the “farmers” of long-term sustainable commercial success), I see the LAMP stack of Linux, Apache, MySQL, and PHP as the big successes of open source projects. (We can debate as to whether C, Cobol, IP Protocol, and others would qualify as well, but I am looking at the commercial models surrounding Hadoop, not the intellectual concept of open source.)

Yet despite Linux’s success in servers, I haven’t seen Linux take the desktop market by storm. On any given day I can hear people grumbling about Office and Outlook, but I don’t see a wholesale switchover to Open Office and Thunderbird. Why? They’re too bloody hard to use. Software designed by committee gives you camels — or elephants — instead of race horses. That’s why some vendors have had success offering consulting and support for FOSS-based solutions. Perhaps there will be a market for [Hadoop consulting](https://www.sandtechnology.com/services/hadoopconsulting/) as well?

Either way, the corporations I talk to tell me they use MySQL to test things, to use for non-mission critical OLTP applications, to get things out in the corporation avoiding the immediacy of having to talk to procurement. When the application rolls into production, they put it on Oracle — it’s safe, stable and great at OLTP.

Enterprises started using Linux when the major vendors started fully supporting it, when it worked no less well than proprietary Unix. But why is Windows still ruling the desktop and why is the Mac gaining share? From a purely commercial perspective I look at open source, I look at Hadoop, and I wonder… Will it be any different? Will it occupy the same cheap, commodity niche that MySQL or Linux occupies now and nothing more?

When we look at Hadoop I see an intellectually stimulating, elegantly conceived concept. What else would you expect from the smart developers who developed it?

What I’m not yet sure about is whether there’s something in the room amid all that poop.