People have been pushing magic cookbooks for software development for decades.  The problem with all the cookbooks is that they miss the important things.  Or maybe the important things can’t be captured in the cookbook.  All of them incorporate elements of the software development life cycle.  Some call it the system development life cycle or application development life cycle.

A hint of why these magic cookbooks are problematical can be found in Cookery Lesson in the introduction to  Software Process Design: Out of the Tar Pit by Jacqueline Holdsworth, 1994, from the International Software Quality Assurance Series.

Cookery Lesson

In his Book of Five Rings, Miyamoto Musashi teaches us that, “The principle of strategy is having one thing, to know ten thousand things.”  It is a teaching I have taken to heart.  Throughout my career I have endeavored to take learning from many fields, and to apply it successfully to software problems.  Somehow it seemed to work well for me, so perhaps you might like to try it too.  Dotted throughout this book you will find an occasional account of an event which occurred in a non-work situation and provided me with insight.  Here is one of them.

Ask people what is the one, single talent they would like to have, and most reply it is to have a musical talent.  For some perverse reason, perhaps because my free time is so short, the one single, talent I have always most craved is to be able to cook.   In my early days, I bought cookbook after cookbook, slavishly following every recipe.  Being constantly disappointed in my efforts, I blamed the cookbook, threw it away and bought another one.  As time went on, it was the weighing scales I found inaccurate, the mixing bowl that was of the wrong material.  What I really needed was a gas oven, a fan-heated electric oven.  And so it went on, until one day, I bought a new cookbook, sight unseen, from a catalogue.  When I sat down to read it, I felt tremendously disappointed, cheated even, because its recipes were like this:

Boiled Cauliflower

It can be mushy or crisp and crunchy.  Pay attention and get it right… the way you want it.

Cauliflower     boiling, salted water     possibly butter    pepper

For the recipe, the cauliflower can be cut in big pieces, but again try to get them of basically uniform size.  Boil, and try a piece after about five minutes.  Butter the cauliflower and season to taste.

I felt so cross, I could hear myself muttering.  ‘What does it mean, pay attention?  How much cauliflower?  What sort of pepper? Possibly butter!’  I had never thought about whether I wanted cauliflower crisp or mushy.  I usually cooked if for however long the cookery book said.  My conversion happened not on the road to Damascus, but at my local market in Cambridge.  Suddenly, everywhere I looked, I saw cauliflowers.  Tiers of creamy white-laced globes stacked in their greenery.  I had never really appreciated ingredients before.  They used to be just things I cooked, and not too well.  But that was when it all started, my love or working with beautiful ingredients, for paying attention, and creating food which now gives delight.

If this book could be described as a recipe book, then it is like this last kind.  My hope is that, after reading it, you will become curious about organizing ourselves to produce software, and instead of relying on other people to give your precise instructions for this or that method, or to use this or that special equipment, you will feel free to see with your own eyes, hear with your own ears, understand with your own mind, and act with your own intelligence in this period of work, which, in the normal course of careers, consumes around 64000 hour of our lifetime.

The steps of the software or system development life cycle were captured and codified and affected the process of software development.

Long ago the government required  standards DOD-STD-2167 and MIL-STD-498 and   IDEF0 , IDEF1.  The standards had lots of documents, but it could be tailored(toss out lots of them).  This was in the days of programmer/analyst who were involved in all the steps of the development life cycle.

The first thing was to talk to the customers.  Find out what they want.  This is requirements gathering.

Better the customers who actually did the day to day work.  Managers would give an idealized overview that would miss the important details.

The more customer viewpoints the better, the more viewpoints the more nuance was captured and the fewer outright misses.

The more ears gathering requirements the better.  Less likely to miss something.

Cobble together the  requirements document and present these to the customers.  Is the lingo correct?   If not the customers set thing right, very quickly. The customers will spot incoherent babbling.

The requirement document was not a monolithic monster.  Every paragraph captured an essential requirement.

During analysis each paragraph would might imply form changes or new forms, report changes or new reports, data structure changes or new data structures, calculation changes or new calculations.

During  analysis and design each requirement would have a section.

  • To support requirement 1.1 needs:
    • Form 1.1
    • Report 1.1
    • Data structures 1.1
    • Calculations 1.1
    • Miscellaneous 1.1
  • To support requirement 1.2 needs:
    • Form 1.2
    • Report 1.2
    • Data structures 1.2
    • Calculations 1.2
    • Miscellaneous 1.2

When explaining to the customers all of the prototyped forms, reports, data structures and calculations things became clear.

Outlining the customer process with forms, reports, data structures and calculations to the customers allowed the customers to see the misunderstandings, and the errors in the thought processes of the programmers.  Ofttimes when explaining ever so lightly the data structures, assumptions that were not obvious in the forms, and reports were made explicit enough that the customers could point out the errors.

Assumptions live at all levels and the abstractions that are used  to hide these details hide  bad assumptions as well as the good.

The customers would tell us what requirements they wanted implemented.  So important requirements were implemented first.

Sometimes the programmers or customers would discover something that allowed other requirements to be abandoned.

Sometimes the customers wanted changes because seeing things live made a lot of  difference.  By closing one eye the changes might fit within the existing requirements.  Sometimes a new requirement was born.

The closer the programmers and customers worked together the better the outcomes.  To paraphrase the many translations of Hemuth von Moltke the Elder

  • Software development is a system of expedients.
  • No requirement ever survives contact with the user.

The  Agile Manifesto and associated  Agile Principles make sense.  But there is always someone trying to turn the ideas into a magic cookbook, or a paint by numbers.  So some of the Agile processes you find in practice look as rigid as its straw man nemesis, the waterfall process.

Agility isn’t determined by the method of documentation.  If you expect things to change, any old documentation methodology can be adapted to your needs.

This guy has a better perspective:

Bye now.