A Methodology of Control of Complexity

John Ousterhout, the author of this book who has built a number of influential systems (Tcl / Tk, Raft, RAMcloud, etc), has ever gave an impressive talk at Google which covers the same theme. In that talk, he asked the audience, "If you had to pick one idea, one concept that is the most important thing in all of computer science, what is it?" and he gave his answer "problem decomposition". Meanwhile he wondered why no course introduced it as it's such important, also no one attempted to teach the skills of being a 10x productive programmer. So he synthesized experiences from teaching and course of career into some insightful and provocative principles on how to (and how not to) design software.
Jumping on the recent trend, I picked up this book based on Alexis's recommendation and skimmed through it over 2 weeks. The book fairly concise at 160 pages, mainly interpret that how to build software incrementally and properly. Overall I agree with the majority of the ideas explained, but I think some others could be misleading or incompatible in specific conditions.
At the beginning, Prof. Ousterhout recommends distinguishing between strategic programming and tactical programming, and refute "Move fast and break things" as company's motto. "Tactical mindset is focused on getting something working, but makes it nearly impossible to produce good system design." On the contrary, developers should do lots of incremental design improvement work over time, later equals never. This is slightly different with Agile Development which focus developers on features, not abstractions, and it encourages developers to put off design decisions in order to produce working software as soon as possible. As he said, "Good design doesn’t really take much longer than quick-and-dirty design, once you know how."
In the first half, the book shows that dealing with complexity is the most important challenge in software design, it is what makes systems hard to build and maintain, and it often makes them slow as well. Complexity is incremental naturally and comes from an accumulation of dependencies and obscurities. The book picks three symptoms of complexity: change amplification, cognitive load, unknown unknowns. Change amplification is a seemingly simple change requires code modifications in many different places, it's easy to prevent by reducing the amount of code that is affected by each design decision. Cognitive load is refers to how much a developer needs to know in order to complete a task, he points out an interesting view, "Complexity is more apparent to readers than to writers. If other people think a piece of code is complex, it is." In consequence, he recommends that we should write codes for readers, who would be ourselves sometimes. Unknown unknowns is that it is not obvious which pieces of code must be modified to complete a task, or what information a developer must have to carry out the task successfully.
The book metions three principles to pull complexity downwards:
Modules should be deep, "The most important issue in designing classes and other modules is to make them deep, so that they have simple interfaces for the common use cases, yet still provide significant functionality." In breif, you should design simplest interface for general-purpose, which also provide a cleaner separation between classes.
Information hiding, "Each module should encapsulate a few pieces of knowledge, which represent design decisions." It's closely related to deep modules. Corresponding to information hiding is information leakage, that means when a design decision is used across multiple modules, coupling them together. To avoid it, you should invest some time on better structure that makes fewer dependencies and deeper interfaces.
Define errors out of existence, becase "have to recover by either trying to revert (hard) or trying to repair and move forward (also hard)", the book advise that the best way to reduce exceptions is by redefining semantics to eliminate error conditions, for exceptions that can’t be defined away, you should look for opportunities to mask them at a low level, so their impact is limited, or aggregate several special-case handlers into a single more generic handler.
In the last part, the book discusses topics such as comments, naming, consistency and performance. That in my opinion makes no difference with some other famous books about management of software and development team, like Clean Code or Peopleware. Afterall, remember the remarkable words while you are building your software:
If you're not making the design better, you are probably making it worse.
The book points out red flags that suggest problems related to each major design issue, which I think is a useful checklist to use during code reviews. When you see a red flag, stop and look for an alternate design that eliminates the problem. The full list of them (the most important ones) are presented as following.
Shallow module
Information leakage
Temporal decomposition
Overexposure
Pass-through methods
Repetition of the same fragments of code
General-specific mixture
Conjoined methods
Non obvious code
Hard to describe
Comment repeats code
Vague names
Hard to pick name
Implementation documentation contaminates interfaces
Interestingly, someone think the book is only suitable for entry level developers, especially graduated students. Meanwhile someone extremely recommend it for experienced developers.
There is no doubt that the book is outstanding once you totally understand it, every developer must read it. If you feel so so, probably you are lack of required experience of complex software management in production.
Besides, some reviewers might argue that author doesn't consider FRP (Functional Reactive Programming) and Object-Oriented is the only pattern of abstractions. As far as I can see, just like Design Patterns by Gang of Four, it's hardly to include all aspects to explain best practice. Afterall, the title of the book is "A Philosophy" rather than "All Philosophy".
Back to the question of "the most important thing", I would say "trade-off".
: )