Among Software Developers there is a distinct range of abilities and level of understanding that ranges from ‘ability to solve business problems’ and then on the other extreme is ‘ability to write compilers’. I’m not saying that if you are capable of writing compilers you cannot solve business problems, because if you work for a tools company that makes it’s money from developing compilers, you’re most likely in the right place.
The issue is, on the majority of software development projects, either in-house development or consulting engagements, you don’t often (if at all) find yourself solving anything technically that has the complexity of developing compilers. This is where the issue is with different developers on this ability/skill scale. For a developer who is entirely focused on solving business problems, and on a consulting project that is the reason you are there, this type of developer will have a hard time understanding why The Compiler Developer is inventing all kinds of wild and wacky technical solutions to problems involving every possible API and open source project they can find. The Compiler Developer on the otherhand has trouble understanding why The Business Problem Solver has no interest in investing time and budget in incorporating all these technical gizmos and doo-dads into the system that from The Business Problem Solver’s point of view just increase the risk to the project.
In reality I believe a project needs to have a good mix of ability to solve business problems and ability to use technology to best solve these problems, and it’s up to the management of the project to keep this balance healthy. If you swing the mix too rich in either direction I think you’ll have a hard time getting anything complete.
One of my most favorite quotes sums up the issues that can be created by leaving Compiler Developers to their own devices:
"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"
– Brian Kernighan, the ‘K’ in ‘K&R’ (Kernighan and Ritchie), co-developer of the C programming language, from http://en.wikiquote.org/wiki/Brian_Kernighan
Kernighan makes a great point, if you have developed something that is so complicated that you have no hope of debugging it, how will anyone else be able to understand it? This is why, unfortunately from the point of view to the Compiler Developers, you need to develop code close to the lowest common denominator of technical abilities on any project, otherwise you are potentially creating a maintenance nightmare for yourself or your client with code so complex that no-one is able to understand, maintain, or fix.