On the Future of Software Development #2 – We are not so special

Let me start my reasoning about how the people-side of the future of software development might look like with two perceptions that form the basis of my argumentation.

Perception 1: Software Development is not that special

I guess we all know those storys of exceptional programmers living on just jolt coke and pizza while developing single handedly a new operating system :-) And we all now the proverbial “diva status” some programmers claim. And we all know how “ordinary people” (our customers) complain about the often introverted natures of programmers and their underdeveloped communication skills.

 

Well, as always there is some truth to such kind of storys. And we can speculate where the reasons lie, what maybe makes dealing with computers so attractive for introverted personalities. But I think much more important is to recognize the underlying attitude of our trade that´s still nurturing these kind of perceptions: Software developers still really think they are something quite special.

 

Remember Donald Knuth´s “The Art of Programming”? Although this book probably sits more on the shelf than it is studied, its title summarizes how generations of programmers view themselves: as artists. In their view software development is an art, a craft, something that is close to composing music. In the center of this picture is the programmer (possibly sitting alone in his cubicle) sculpting an ingenious algorithm out of the clay of the formal language of his choice.

 

Well, a nice picture and I sometime indulge in feeling like that when pondering the intricacies of parallel programming or the like. But it´s not reality! Maybe it has been like that 20 years ago, but we cannot hold up such an attitude any longer.

 

20 years ago a programmer or software developer or software engineer had been a highly trained professional in a narrow field of expertise: software development. This distinguished him from carpenters and aircraft engineers.

 

On the other hand, once highly trained in this trade, the programmer felt ready to tackle any and all problems. Regardless if he was asked to develop a compiler, a book keeping application or painting tool he never would have declined such a request. He felt competent to wield the state-of-the-art tools and materials to produce a satisfying result. He just needed complete requirements and enough time to wreck his brain – which was his most important tool.

 

Back than this might have been a valid attitude, but it got carried over to the present. And that´s plain wrong and makes it very difficult for developers today to produce high quality and get job satisfaction.

 

20 years ago software development was special in that it had few tools and materials and mostly relied on the creativity of single developers with deep knowledge in theoretical concepts (like “algorithms and data structures”).

 

Also software seemed and still seems to be so different from anything else humans produce. This adds to the trade´s attitude of being special and unique. Software is so abstract, so highly formal/structured, so ephemeral. Software developers were the digital guys in an analog world. Bits and Bytes made ordinary people gape in awe. Programmers were the heros able to tame the computer beast by wielding their formal language whip. Wow!

 

But, hey, let´s get real! Our world of 2005 is digital. Music is digital since we switched to CDs in the mid 1980s. Video has become digital. Telephoning is the last bastion about to fall when finally VoIP is taking off on the consumer market.

 

Computers in some form are everywhere and RFID will add to the pervasiveness. And kids get aquaintant with programming in elementary school. Software development has lost its nimbus of a secret art.

 

But how about our products? They are digital and can easily be mass produced. (Just copy that file.) And they are highly individual. And they are made on and for changing requirements and a long maintenance life. Isn´t that quite unique?

 

Well, I wouldn´t say so. Although analogies are a difficult thing and usually don´t explain everything, let me try some:

 

Are there no other products that need long maintenance? Sure, there are. Chemical plants, aircrafts, and even cars are build in a relevatively short time and need to be maintained over a very long time (up to decades). Or even take a city as example: You don´t build a city and then leave it to itself. Rather it is constantly changing, buildings are torn down, new ones are build, subway tunnels are dug etc. Some structures are more volatile than others. But over the decades or even the centuries a city changes massively without losing its basic character and while always fulfilling its main purpose: offer an infrastructure for the inhabitants. My conclusion: Mankind or engineering in general has a lot experience with long lasting structures that need to undergo quite some changes during their lifetime. Software is not special in that regard.

 

Are there no other products that are mass produced but always unique? Sure, there are. Take roller shutters for example. Each roller shutter usually is tailored to the exact measures of a window/opening of a house. Or – an analogy I even like more – take a movie as an example. Each movie is a unique product. Hollywood produces 400 movies each year, the Indian Bollywood even 800 each year. And despite its uniqueness (and often changing requirements; think of producers as divas ;-) it is an industrial product and a result of a coordinated effort of hundreds of individuals.

 

So I´d say, long lifetime with high maintenance and uniqueness are not really attributes of software development, which justify a special status or attitude. Also the easyness of multiplication is not unique. The music and video industry are facing the same problem. And print is struggling with is since the invention of photo copying machines.

 

But what about the ever changing requirements? The waterfall model failed for software development, but it works for civil engineering. Yes, the requirements seem to change much more often during the manufacturing of software compared for example to a movie. But does that make software development uncomparable to other manufacturing processes? I don´t think so. Because not the magnitude of volatility of requirements is crucial for a comparison, but whether the tools, materials, models, processes, education etc. match a given volatilty.

 

And therein lies the crux, I´d say! Don´t try to compare apples with pairs. Don´t try to compare a mature trade like “building skyscrapers” or “manufacturing cars” with a comparatively young (or immature trade). So, you should not take a rediculous comparison for a prove of uniqueness.

 

When comparing software development to some other industry/trade, first ask, if software development can match the other side´s maturity and in how far that could affect the outcome of the comparison. My opinion: Software development is still after some 50 years pretty immature – but not as unique as most people think. (The uniqueness of software development, though, might lie in its stubborn self assurance of its uniqueness. Remindes me a bit of autism.)

 

But, now, finally, what about the lack of natural laws in software development? Yes, that´s true. That poses a certain difficulty and accounts for some uniqueness. When planning a building or a CPU, the effects/stability of certain designs can be assessed by using mathematical formulas or by simulation based on natural laws (i.e. again mathematics). Before even digging the smallest hole you know if a bridge will be able to carry the trains and cars rolling over it later. That´s because you know the exact parameters of the ground it is build on, the stones, the steel, the concrete etc. which are going to be used in building it. For software, though, you cannot just compute its stability or correctness or quality. Contrary to most other trades you need to constantly monitor and actively asses it (that´s called “testing”) or model scenarios when in doubt (that´s called “prototyping”). And since each software is unique, you are never done with testing.

 

But does this uniqueness justify the software development´s trade attitude of “Oh, boy, are we special!”? No, I doubt it. It is not so important how we determine correctness of our products. It is important to have a mature methodology for doing so that is adequate for our tools and materials. And likewise it is not so important, that software development differs so much and lacks “mathematical precision”. We must not feel inferior because of that. We just do it differently. Correctness is determined in different ways when building skyscrapers, finding a cure for a disease, coming to a judgement in a trial. So why can´t we be different – and still be comparable to other trades?

 

So when comparing software development to other trades the question is not, whether we produce similar products. Rather we should ask, do the processes differ fundamentally? We should look at mature and established industry and see what they consist of: there are tools, materials, process phases (e.g. planning, manufacturing, maintenance), division of labor, quality assurance, professions, education etc.

 

Then we should compare our findings with what we have in software development. And we can only realize: all of this is present in our trade. So software development does not fundamentally differ from other industries!

 

But if we don´t differ fundamentally, why do we have a hard time liking software development to other trades? Because the level of maturity of our tools, materials, process phases, division of labor, quality assurance, professions, education is so different from their´s. And the level of maturity does not even always match the complexity of the problems to solve.

 

Don´t get me wrong, there are good tools at our hands and there are excellent programmers. But still we can and should learn a lot from other much more mature industries. UML is an approach to learn documentation and visual modelling from civil engineering or electrical engineering or processor design. Software Factories now is an approach to learn from mass manufacturers about tool specialization and manufacturing. Now my question is: What can we learn from other industries in terms of process organization, education, quality assurance, divison of labor?

 

And if our trade is not unique, neither is our profession as software developers. We should not deem ourselves as artists or divas or very special in general. We are ordinary hard working people in a demanding business with some very specific challenges. Not more, not less.

 

The only thing that maybe makes us special is, we need to deal with a lot more change than other professions. But that will be the topic of the next installment of this series.

 

[See here for an overview of all postings of the series on the future of software development.]

 

PS: If you doubt the immaturity of the software development trade, think a minute about the following:

 

-Why do we still store the majority of code in text files which are hard to manipulate?

-Why is deployment, i.e. handing the product to the customer, such a pain?

-Why do we still try to solve all those different problems with just a handful of different languages?

-Why do so many software projects fail (in comparison to building houses or producing movies)?

 

Comments

# re: On the Future of Software Development #2 – We are not so special

Monday, March 21, 2005 11:36 AM by Mathias Trapp

"Before even digging the smallest hole you know if a bridge will be able to carry the trains and cars rolling over it later. That´s because you know the exact parameters of the ground it is build on, the stones, the steel, the concrete etc. which are going to be used in building it."

We should also consider that all the other engineering trades are able to take advantage of built-in flexibility, which software development hasn't. While planning for a bridge, they oversize the strongness of the steel or of the concrete by a factor of 1.30 to 1.50 (or in critical buildings by an even larger factor). With programs you only have the choice of true and false - there is no buffer, if your application is put under unforseen stress.

Mathias Trapp
Software Engineer
Germany

# re: On the Future of Software Development #2 – We are not so special

Monday, March 21, 2005 12:17 PM by Steve Day

forgot to sign my comments.

sday@dayhaus.com

# re: On the Future of Software Development #2 – We are not so special

Monday, March 21, 2005 12:36 PM by Ralf

@Mathias: Some flexibility we can add, too. For example, design domain logic as a stateless component, even though you will use a C/S architecture without distribution first. Later on you then can much easier move to an app server.

Or take a plug-in design. Maybe in the beginning you don´t need logic pluggable. But still you design a plug-in interface.

# re: On the Future of Software Development #2 – We are not so special

Monday, March 21, 2005 12:47 PM by Ralf

@Steve: Sorry, I don´t agree with all of your explanations:

deployment: I´d counter by saying, customers are paying more today, than they´d, if deployment were easier. They just don´t know, they are in a position to demand less trouble. Or alternatively: Platform manufacturers slowly are realizing that ease of deployment is important and take steps to improve it. But slowly. XCOPY deployment of .NET assemblies is a major advantage over COM component registration. ClickOnce is another step. But still... there´s a long way to go.

languages: There contrary will happen: we will see more and more specialized languages (domain specific languages). There might be some convergence in GPLs, but all in all we see more languages, because languages are tools. And as an industry matures it specializes its tools.

failed projects: So if anybody can pick up a compiler and call himself a "software engineer", than there is something wrong. I don´t mean compilers should become more difficult to use ;-) Rather a customer (or employer) should have better means to asses the professional skill level of a programmer.

movies: Selling moviers might be simple. But meeting the taste of the potential audience is difficult (despite stars and happy ends).

# re: On the Future of Software Development #2 – We are not so special

Wednesday, March 23, 2005 7:33 PM by Ed Daniel

Software has a long way to go in order to get to where consumer electronics currently is. However at that point what is prevalent in the marketplace beyond the spec and feature sheets is the manner in which that product is built, i.e. it's design in relation to how we interact with it.

I am in love with the thought of software as 'art'.

I find Andy Warhol's quote about business and art inspiring in this respect:

"Making money is art and working is art and good business is the best art"

Architects (not software) are prized for their approach to design. That is the human element that touches all of us, we (general public) don't care for the resilience of the structure, its functionality i.e. 'fit-for-purpose', it's use of material in the most efficient manner because we assume/expect that to be so.

What is prized is the end result and it's effect (visual).

While the software industry continues to mature at an ever increasing pace it is the pleasure a user derives from your software that makes it great. In turn that makes you, the software author, special.

Yes, anyone can code now, mainly because the knowledge is there for the taking - it's how we use our knowledge and devise our approach that sorts the engineers from the artists.

# re: On the Future of Software Development #2 – We are not so special

Thursday, March 24, 2005 2:50 AM by Ralf

@Ed: I feel with you, when you say, you´re "in love with the thought of software as an 'art'." I´d love it to continue to be an art - but I think it cannot (if it ever was).

And I don´t agree with Andy Warhol. How he´s using the term "art" (and how Joseph Beuys used it) is not how I´d would use it. Maybe that´s where we differ. Business or software is not in the same league like painting, sculpting, acting, composing and performing music. Neither is architecture.

However what I see perfectly fit for software is a concept of "mastery" like it´s known in the western world. Or take "shu ha ri" (3 stages on the way to becoming a master) from the far east.

Certainly we have and need masters of our trade! And there should be a roadmap, an agreement on skillset and criteria for recognizing a true master.

As for design: I love good design (from coffee cups to cars and books :-). And I recommend "The Design of Everyday Things" as reading for any software developer. But a need for good design does not make a trade an art form.