"Highly Recommended: This book provides readers with an informal, pragmatic approach to software development. It was well worth studying and has proved enlightening...This is a good book that should be read by intermediate to advanced C++ developers. If you don't have time to read it, then at least have a quick look through. You're bound to find a chapter that attracts your attention and enlightens you." -- Ian Bruntlett, Overload
"If you want to broaden your perspective and willing to risk facing a world in which OOP is not always the best, then you should take the time to read this book. I suspect that having done so you will want to hang on to your copy so as to read it again in a leisurely fashion... Finally, if you ever get a chance to listen to James Coplien speak grab it with both hands, he is one of those exceptional speakers that set standards that most of us can only aspire to." -- Francis Glassborow, of the Association of C and C++ Users (ACCU), in C Vu January, 1999
"Jim Coplien is a recognized authority in several areas of software development: C++ programming, the object paradigm, patterns, and the organizational aspects of software development projects. His latest book combines his knowledge of all these topics... The book excels because of the author's broad knowledge... Multi-Paradigm Design in C++ is a great book crammed with elaborate and compelling ideas. Reading it will change your perception of software development. " -- Angelika Langer and Klaus Kreft, The Development Exchange
"May well be one of the most important books ever written in the software field... treats an extremely complex topic in an extremely complex manner" -- The Software Practitioner
"This may be one of the most important books ever written in the software field." ... "Who would believe that behind the covers of a book that announces it is about 'multi-paradigm' something-or-other, is perhaps the definitive book on domain analysis and engineering. - how to choose among competing methodologies - distinguishing between the problem and its solution" ... "The software engineering field in the future must focus heavily not just on applications but on their families." -- Robert L. Glass, in Software Practitioner, May 1999
I have rarely invested as much energy in any endeavor as in naming this book. As the manuscript evolved, its title evolved to emphasize one or two concepts at any given time from the set of basic elements Domain, Engineering, Multi-Paradigm, Analysis, Design, Programming, and C++. The publisher was afraid that the unfamiliar term "domain engineering" would fail to engage the target market. One of the reviewers, Tim Budd, was concerned about confusion between his use of "multi-paradigm" and the way the term is used in this book. I was concerned about using terms such as "analysis" because of my desire to put the book into the hands of everyday programmers, whose problems it strives to address. Tim Budd graciously offered that our discipline is diverse enough to accommodate a broad spectrum of definitions for "multi-paradigm," and I insisted on a title that emphasized the role of the programmer and not that of the methodologist. That led to a happy convergence on the current title, Multi-Paradigm Design for C++.
I never considered titles containing the words pattern, object, CORBA, component, or Java. Multi-paradigm design tries to dig deeper than any single technology or technique to address fundamental questions of software abstraction and design. What is a paradigm? What is the relationship between analysis, design, and implementation? These questions go to the foundations of abstraction that underlie the basic paradigms of programming.
One of the most basic questions is, what is abstraction? Abstraction is one of the key tools of software design; it is necessary for managing the immense and ever-growing complexity of computer systems. The common answer to this question usually has something to do with objects, thereby reflecting the large body of literature and tools that have emerged over the past decade or two to support object-oriented techniques. But this response ignores common design structures that programmers use every day and that are not object-oriented: templates, families of overloaded functions, modules, generic functions, and others. Such use is particularly common in the C++ community, though it is by no means unique to that community.
There are principles of abstraction common to all of these techniques. Each technique is a different way of grouping abstractions according to properties they share, including regularities in the way individual entities vary from each other. To some, commonality captures the recurring external properties of a system that are germane to its domain. To others, commonality helps regularize implicit structure that analysis uncovers in the recurring solutions for a domain. Multi-paradigm design honors both perspectives. For example, the technique called object-oriented design groups objects into classes that characterize the common structure and behaviors of those objects. It groups classes into hierarchies or graphs that reflect commonality in structure and behavior, while at the same time allowing for regular variations in structure and in the algorithms that implement a given behavior. One can describe templates using a different characterization of commonality and variation. Commonality and variation provide a broad, simple model of abstraction, broader than objects and classes and broad enough to handle most design and programming techniques.
Commonality and variation aren't new to software design models. Parnas's concept of software families Parnas1976 goes back at least two decades. Families are collections of software elements related by their commonalities, with individual family members differentiated by their variations. The design ideas that have emerged from software families have often found expression in widely accepted programming languages; good examples are modules, classes and objects, and generic constructs. The work of Lai and Weiss on environments for application-specific languages takes this idea to its limits Weiss1999. The so-called analysis activities that focus on the discovery of software families and the so-called coding activities that focus on how to express these abstractions have always been closely intertwined. Multi-paradigm design explicitly recognizes the close tie between language, design, and domain structure and the way they express commonality and variation.
We discover software families in an activity called domain analysis, which is another field with a long history Neighbors1980. Software reuse was the original goal of domain analysis, and this goal fits nicely with software families. Multi-paradigm design explicitly focuses on issues that are important for reuse. To help the designer think about adapting software to a spectrum of anticipated markets, multi-paradigm design explicitly separates commonalities--assumptions that don't change--from variabilities--assumptions that do change. We strive for domain analysis, not just analysis. We design families of abstractions, not just abstractions. Done well, this approach to design leads in the long term to easier maintenance (if we predict the variabilities well) and to a more resilient architecture (we don't have to dig up the foundations every time we make a change). Of course, multi-paradigm development is just one tool that helps support the technical end of reuse. Effective reuse can happen only in the larger context of organizational issues, marketing issues, and software economics.
We use these foundations of commonality and variation to formalize the concept of paradigm. A paradigm, as the term is popularly used in contemporary software design, is a way of organizing system abstractions around properties of commonality and variation. The object paradigm organizes systems around abstractions based on commonality in structure and behavior and variation in structure and algorithm. The template paradigm is based on structural commonality across family members, with variations explicitly factored out into template parameters. Overloaded functions form families whose members share the same name and semantics, and in which each family member is differentiated by its formal parameter types.
C++ is a programming language that supports multiple paradigms: classes, overloaded functions, templates, modules, ordinary procedural programming, and others. Bjarne Stroustrup, the creator of C++, intended it that way. Most programmers use the C++ features that go beyond objects (though some abuse them to excess and others force designs into an object-oriented mold when they should be using more natural expressions of design provided by other language features). The powerful template code of John Barton and Lee Nackman Barton1994 is perhaps the height of tasteful multi-paradigm design.
Even though Stroustrup designated C++ as a multi-paradigm language, there have been no serious attempts to create a design method suited to the richness of C++ features. And though C++ provides a particularly rich and crisp example of multi-paradigm programming, the opportunity for multi-paradigm development generalizes to other programming languages. There is a gap between the current design literature and the intended use of C++ features that is reflected in current practice. This book bridges that gap, using simple notations and vocabulary to help developers combine multiple paradigms instructively.
During a lecture I gave at DePaul University in September 1995, the department chair, Dr. Helmut Epp, suggested the term meta-design for this work because its first concern is to identify design techniques suitable to the domain for which software is being developed. That is a useful perspective on the approach taken in this book and in fact describes how most developers approach design. One must first decide what paradigms to use; then one can apply the rules and tools of each paradigm for the system partitions well-suited to their use. This concern is the domain not only of the system architect and designer, but also of the everyday programmer.
Deciding what paradigm to use is one matter; having tools to express the abstractions of a given paradigm is another. We can analyze the application domain using principles of commonality and variation to divide it into subdomains, each of which may be suitable for design under a specific paradigm. This partitioning occurs during a development phase commonly called analysis. However, it is better thought of as an early phase of design because it tries to create abstractions that the implementation technology can express. Not all implementation tools (programming languages and other tools such as application generators) can express all paradigms. For this reason, it's important to do a domain analysis not only of the application domain, but also of the solution domain. Multi-paradigm design makes this an explicit activity. Solution domain analysis is another facet of the "meta-design" nature of multi-paradigm design.
There are many things this book is not. It is not a comprehensive design method, software development life cycle model, or turn-the-crank approach to design. Most good new designs are variants of old designs that have worked; it's rare that we face completely new or unique software problems. It would be inappropriate and a waste of time to apply the notations and techniques of this book to every module in a new system. But we should be armed to face new problems when they arise so that we can discover the structure in them and carry that understanding into design and implementation. Furthermore, the notations and techniques of multi-paradigm design provide a uniform way to document designs that augment object-oriented techniques with other paradigms.
Multi-paradigm design is a craft that is neither fully an art nor fully a rigorous discipline. This book presents notations, diagrams, and design models to support the developer's thought process. As is true with all such formalisms, there is always the temptation to get caught up in them for their own sake. Multi-paradigm design is a collection of activities that produce an architecture, and architecture is about relationships between pieces. But architecture is also about utility and aesthetics--properties of good software that elude most methods. Good taste has a key place here, and I don't intend the techniques to fuel automated design or coding. Good taste comes in part from experience and in part from good insight. For that reason, this isn't an entry-level book, either. Readers should have a year or two of experience using C++ doing object-oriented (at least) programming in a substantial system.
Used with common sense, these techniques complement good human judgment and experience. If you find that by applying these techniques, you arrive at a design you neither like nor understand, then don't use the design. The techniques are a tool, not a mandate. But all readers should take one thing away from this book: The object paradigm, or any other paradigm, is just one set of useful paradigms, and design must express structures more broadly than any single paradigm can.
The Book's Organization
Each chapter builds on the ones before it to build new concepts and increase the reader's understanding of domain engineering and multi-paradigm techniques. Most readers will read chapters sequentially and return to specific chapters as reference material.
Chapter 1 through 7 are the foundational chapters that lay the groundwork for domain engineering. Chapter 1 introduces vocabulary, motivates the need for multi-paradigm design, and lays high-level foundations for domain engineering. Chapter 2 and Chapter 3 cover commonality and variability analysis, respectively. The two concepts are used together in application, but each is presented in its own right both for simplicity of presentation and to emphasize its language subtleties. Chapter 3 introduces the central concepts of positive and negative variability, themes that are important to clean designs and to understanding how domain engineering interacts with commonly used design patterns. Chapter 4 explains how to use domain analysis to find abstractions in the application domain; it builds on the techniques of the preceding chapters. Chapter 5 demonstrates how the principles of domain engineering can be used as a basis for the abstraction techniques of the object paradigm. Chapter 6 is an important chapter, both because it applies "analysis" techniques in an unconventional way--to characterize the solution domain--and because it places C++ constructs in a formal framework that form the basis of commonality and variability analysis. Chapter 7 ties together all of the preceding chapters into a coherent framework for thinking about design. It introduces a taxonomy of design problems that have varying degrees of structural complexity. It also presents a high-level set of activities that guide good design and that could be the basis for a method based on domain engineering and multi-paradigm design techniques. The chapter deals with the simple case in which each domain can be developed largely independently of others in a single paradigm. Chapter 8 goes a step further, investigating structurally complex designs in which simple divide-and-conquer techniques don't prevail. It presents examples of such recursive, structurally complex designs and presents heuristics for "breaking the recursion." Chapter 9 is an addendum that shows the relationship between popular design patterns and domain engineering. The parallels will surprise many readers, and provide new insights into both patterns and domain engineering, as well as the relationship between them. Though the chapter isn't central to understanding domain engineering, it is crucial to understanding contemporary design practices that incorporate patterns together with objects and other paradigms, with or without domain engineering.
Class diagrams in the book follow the Unified Modeling Language (UML) notation Fowler+1997. Acknowledgments
Many thanks to all of those who, through their dialogues, comments, and feedback, improved the quality of the manuscript. Just van den Broecke, Frank Buschmann, Paul Chisholm, Russell Corfman, David Cuka, Cay Horstman, Andrew Klein, Andrew Koenig, Stan Lippman, Tom Lyons, Lee Nackman, Brett Schuchert, Larry Schutte, Herb Sutter, Steve Vinoski, and David Weiss all provided review comments at many levels, from the highest level of conceptual murkiness to the smallest C++ detail. I appreciate them very much for their efforts. I also want to thank my tireless and patient editor, Debbie Lafferty, who also worked with me on my first book. She has been a true joy to work with. Special thanks also go to Jacquelyn Young, my production editor; to Laura Michaels, the book's copy editor; and to Kim Arney, the book's compositor. Discussions with Lalita Jagadeesan inspired me to create useful examples. A special thanks to Tom Stone, the acquisition editor, for his early advice to me and his early enthusiasm about this book inside Addison-Wesley. A special thanks to Andrew Klein for help with the UML diagrams. And last, a special thanks to my Bell Labs management and colleagues, especially to David Weiss, for sharing their work with me and providing support and encouragement.