According to this article, netbook manufacturers are temporarily putting production of the next lines of their netbooks to go on sale for the end of the year on hold, waiting to see what Apple comes up with regarding their ‘tablet’ device. According to the article, an unnamed industry analyst who has seen the device, says it will be announced in September and released in November this year.
Current software technologies on my radar: alternatives to RDBMSes and extreme scalability solutions
From working on new systems for what I call ‘regular’ businesses, it’s rare nowdays (in the past 6-7 years at least ) to work on anything that isn’t web-enabled with some mix of HTML/CSS/Javascript, implemented using Java server-side, using some RDBMS for data storage and retrieval. What I find interesting is the current mix of technologies being used by some of the more popular web-presences (Google, Yahoo, MySpace, Facebook, Twitter, Flickr etc) that are using technologies other than this standard technology stack to address different problems of running an online service with potentially millions of users, or in the case of Google and Yahoo, providing online search engines with searchable indexes containing millions of websites.
The current technologies that I see showing up more and more include some of the following:
- The MapReduce algorithm was made popular by Google using it to build their search indexes, but now the approach seems to be showing up as the solution to solve everyone’s massively parallel data processing needs. Apache Hadoop is a Java implementation of a MapReduce framework, and checking out their page listing their users (several pages long), the impressive list of usage examples ranges from clusters of a few machines to clusters with 1000s of machines and 1000s of CPUs with upto 100 terrabytes of data. Those stats are something that I’ve personally never come across implementing typical business systems, even ones with 1000s of users – definitely some interesting problems being solved in these areas.
- Memcached shows up when reading about the implementation of most online services, in the context of avoiding database hits and keeping as much of the frequently accessed data in memory as possible as a performance optimization.
- There’s no shortage of ‘my tried and tested old school technology is still better than your new fangled web technology’, and this article commenting on some misconceptions about what MapReduce is and is not is interesting reading too.
What’s clear is there’s plenty going on out on the edge of massive multi user online services right now, and plenty of interesting developments to keep an eye on.
Maintainability: if you can only just understand what you developed, no-one else will be able to
Among Software Developers there is a distinct range of abilities and level of understanding that ranges from ‘ability to solve business problems’ and then on the other extreme is ‘ability to write compilers’. I’m not saying that if you are capable of writing compilers you cannot solve business problems, because if you work for a tools company that makes it’s money from developing compilers, you’re most likely in the right place.
The issue is, on the majority of software development projects, either in-house development or consulting engagements, you don’t often (if at all) find yourself solving anything technically that has the complexity of developing compilers. This is where the issue is with different developers on this ability/skill scale. For a developer who is entirely focused on solving business problems, and on a consulting project that is the reason you are there, this type of developer will have a hard time understanding why The Compiler Developer is inventing all kinds of wild and wacky technical solutions to problems involving every possible API and open source project they can find. The Compiler Developer on the otherhand has trouble understanding why The Business Problem Solver has no interest in investing time and budget in incorporating all these technical gizmos and doo-dads into the system that from The Business Problem Solver’s point of view just increase the risk to the project.
In reality I believe a project needs to have a good mix of ability to solve business problems and ability to use technology to best solve these problems, and it’s up to the management of the project to keep this balance healthy. If you swing the mix too rich in either direction I think you’ll have a hard time getting anything complete.
One of my most favorite quotes sums up the issues that can be created by leaving Compiler Developers to their own devices:
"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"
– Brian Kernighan, the ‘K’ in ‘K&R’ (Kernighan and Ritchie), co-developer of the C programming language, from http://en.wikiquote.org/wiki/Brian_Kernighan
Kernighan makes a great point, if you have developed something that is so complicated that you have no hope of debugging it, how will anyone else be able to understand it? This is why, unfortunately from the point of view to the Compiler Developers, you need to develop code close to the lowest common denominator of technical abilities on any project, otherwise you are potentially creating a maintenance nightmare for yourself or your client with code so complex that no-one is able to understand, maintain, or fix.
Apple tablet on the way for 2010
It’s been rumored for a while that there was a tablet-like device coming from Apple, and the latest rumors from sources say manufactures will have the device ready for early next year. The device is expected to cost somewhere between an iPod Touch and the low end MacBook, and be largely based around the iPod Touch.