Is the RDBMS on the way out?

If you read a lot of blogs and IT industry commentary online, at some point this year you will have come across an article from someone talking about the ‘death of the RDBMS’. Even though RDBMSs from the major vendors (Oracle, IBM, Microsoft) are still very much a corner stone to most large-scale business systems today, you can’t help but notice that a lot of new systems being built that address problems of enabling massive online services are not using RDBMSs any more. Huh? No, they’re using other data storage mechanisms that are turning their back on the past 30 years or so of experience built up from using RDBMSs, or is it that we’re actually reaching the point where we’re realizing maybe RDBMSs are not actually the best/most appropriate way of storing and retrieving data for all scenarios?

Most of the alternatives to RDBMSs that seem to be gaining traction seem to have roots in Google’s approach of BigTable and Google File System, where information is stored in a heavily denormalized format with tables having thousands of columns. Rather than breaking data down into relationships an separate tables, the main feature of this approach is simply representing data as key/value pairs. You have a key, you have a data value. Simple.

Here’s a collection of some articles I’ve come across discussing the current trend of RDBMSs going out of favor – here, here and here.

Duct Tape Programmers & Smart Developers

I love Joel Spolsky, he just says it as it is. His comments here about Duct Tape programmers are spot on.

You need developers like this on any project – see the issues, find solutions, fix issues, get stuff done. And simply, in a way everyone else can understand. I love the comment about some technical approaches being easy to understand if you are Spock. That puts a lot of things in perspective. Being smart just for the sake of being smart does not get things done, does not help your fellow developers on the team, and usually does not help everyone get the job done and the product out of the door. What a lot of smart developers fail to realize is that there is a need to aim the level of complexity on any project to the level of the least common denominator. Think about it. If you are busy architecting and dictating ‘the way forward’ using terms and technologies that only a small number of developers on your project can understand, how does that help move the project forward to meet it’s end goal? It doesn’t. What smart developers need to realize is that being smart in order to help a project reach it’s end goal means they have to use their smartness to find the easiest, most appropriate solution that every developer can understand. Unless you can find this common ground you are not going to leverage your whole team. Think about it. If you have a team of 50 developers and you are proposing an architectural solution that only 5 developers understand and can use, this is pointless. This is not an appropriate solutions, regardless of how smart or proud you are of thinking of the idea.

It takes some smart developers a while to come to this realization, and some never get it at all. The ones that do become very valuable resources on projects because they help the team as a whole get things done. The ones that don’t get it, well, they’re destined to become astronaut architects and live in the clouds.