Progress in Software Technology

Programmers of the first computers quickly had to discover the spaghetti code problem -- programs turned by jump instructions and data-dependent decisions into totally obscure tangles of control paths -- so that beyond about a few hundred commands programs ceased to be evolvable. Since then, computer science has made tremendous progress, restructuring systems such as to be more transparent and evolvable. Today, systems consisting of millions of lines of code are widespread and are worked upon by thousands of programmers, in some cases being dispersed over the whole globe.

Some pervasive general trends have lead and are leading to profound improvements in software technology, conspiring to form more and more powerful architecture:

• More principled ways of programming.
• Reduction of the level of arbitrary decisions, which lead to lack of coherence and to the necessity for extensive documentation.
• Reduction in the number of degrees of freedom in a programming system, leaving only those that are inherent in the application domain.
• Decisions once for all.
• Re-use of structures.
• Lining systems up behind high-level schemas.
• Adaptation to the perspective of the human programmer, e.g., by assembling functionally related variables within the individual programmer´s limited attention span, while protecting them from interference by other programmers.

• Data types: different kinds of numbers, symbols, pointers, instructions, lists, structured arrays, graphs and more.
• Systematic treatment of variable scope.
• Hierarchies of sub-routines.
• Patterns, templates or classes, and instantiation and inheritance relations.
• High-level programming languages, adapted, for instance, to numerical scientific, financial or algebraic computing, or to text processing, industrial process control, computer graphics, web design, and many more.

• Operating systems: Management of many hardware and software utilities, of several users and of many process streams.
• The desktop paradigm.
• Enterprise software.

Whereas originally a single programmer designed an algorithm for a well-defined problem and an algorithm was typically executed without interruption, the modern computer has to deal with rapidly changing and unforeseen situations. Thus, computing was originally dominated by imperative languages and main programs, which rather rigidly determine the sequence of action. This style naturally corresponds to the sequential control of the von Neumann architecture. Now, a growing theme in computing is the co-ordination of otherwise independent sub-processes. Correspondingly, emphasis is placed more and more on concepts inspired by the metaphors of societal, neural, and more generally biological, organization, imperative languages being replaced by agent-based and similiar systems, determinism giving way to probabilistic, adaptive and evolutionary mechanims.

The software crisis can only be resolved if these trends are pursued much more vigorously than they are today, essentially in the form of a paradigm change, fundamentally changing the terms of the algorithmic division of labor by loading creative infrastructure into the computer, such that man can ultimately relinquish control of detailed computer processes to general mechanisms of organization and restrict his role to high-level definition of the goals to be pursued.

Just as life is the phenomenon of organization plus a large collection of special tricks and mechanisms, computing structures are architectures plus specific utilities and algorithms. Specific algorithms will eternally be a matter of specific invention. But the phenomenon of goal-oriented organization should be treated once for all, turning the current black art of software design into a science.