This is about programming or at least solutions materialized using computers and software. Apply it to other areas at your own risk.

I know how to program. Its not what I do for a living anymore.

For many problems within my field of expertise I can give a rough estimate, and with rough I mean that I get it right within a factor of 2. Ask me the price for building something and I can tell you 8h, 16h, 40h, 80h, 160h, 400h, 800h, 1600h… you get it. The estimate also includes generally what tools to use (C, ASP.NET, Linux+bash, LAMP).

My observation: many people – professionals – cant get this right.

I believe it has to do with (a lack of) intuition for complexity:

• Software-related problems are much more non-linear that most real-world-problems.
• For software, the hardest part of a problem tends to determine the hardness of the whole problem
• Solving two problems together, or merging the solutions of two problems, tends to generate a much more than twice-the-size challenge

The result: people who fail to thoroughly understand the above can not reasonably (within a factor of 2) estimate the cost of a software solution.

There are simple solutions, and complex solutions – often to the same problem. Once I watched news about the unmanned subway trains in Copenhagen. They interviewed the responsible manager and asked him: is it not dangerous with no one to brake the train? He answered: no it is safe, we have a very complex system monitoring the trains and the safety. Horror – I thought! I wish he said they had a very SIMPLE system. Because to me

Simple means many people are able to understand and verify the correctness of the system.
Complex means few people (and maybe no single person) are able to understand and verify the correctness of the system.

Sometimes I draw this matrix:

In Einsteins words: Everything should be made as simple as possible, but not simpler.

Where is this going? Well, my question today is:

Why do people choose to attempt unnecessarily complex solutions to problems?

In other words: Why do so many computer professionals suck so bad at their job?

First lets just despise those who intentionally make money by selling more hours than necessary. But those with good intent, who are just bad?

I think they simply fail to understand complexity, as I mentioned before, so they simply can not tell a simple solution from a complex solution. Obvious.

Whats worse, I believe that they prefer complex solutions, based on the real-world fact that more-is-more.

Whats even worse, and this is just my conspiracy theory, is that they prefer complex solutions because; not understanding a complex thing is normal, but not understanding a simple thing is obviously stupid. Thus, stupid people are drawn to complicated stuff to not reveal their stupidity. It is just a survival strategy in the job market, to level the field a bit.