The Coming Software Apocalypse in The Atlantic calls attention to a serious problem in the software industry: software has become too complex for total individual comprehension and we have failed to adapt the engineering process accordingly.
This flexibility is software’s miracle, and its curse. Because it can be changed cheaply, software is constantly changed; and because it’s unmoored from anything physical—a program that is a thousand times more complex than another takes up the same actual space—it tends to grow without bound. “The problem,” Leveson wrote in a book, “is that we are attempting to build systems that are beyond our ability to intellectually manage.”
An outage of Instagram, Snapchat or some other social networking service caused by engineering failures will hurt little more than reputation and ego, but software systems that control aircraft, finance or utilities are less fault tolerant—the cost of failure could be death. Brett Victor observes languages and approaches have evolved little in the last half century.
Our current conception of what a computer program is, is derived straight from Fortran and ALGOL in the late ’50s. Those languages were designed for punch cards.
To illustrate, I learnt from The C Programming Language second edition (published 1988) at university, my father learnt from the first edition (published 1978). Paradigms and techniques need not to be rejected simply for their age, yet limitations have become apparent in their application to complex critical systems. In fact these limitations have been visible for a number of decades and new approaches are gaining popularity like TLA. Less esoteric approaches are already very popular like stateless programming.
I think the most important observation in The Atlantic’s article is that software systems must be plannedi—planned when first built and planned when extended. Leslie Lamport writes:
Architects draw detailed plans before a brick is laid or a nail is hammered nut few programmers write even a rough sketch of what their programs will do before they start coding.