Nobody Cares About Your Build

Every software system, from simple Python packages to huge enterprise-grade systems spanning massive clusters, has a build—a set of steps that must be followed to go from a source tree or a checked-out project to a ready-to-use build product. A build system's job is to automate these steps.

Build systems are critical to software development.

They're also one of the most common avoidable engineering failures.

A reliable, comfortable build system has measurable benefits for software development. Being able to build a testable, deployable system at any point during development lets the team test more frequently. Frequent testing isolates bugs and integration problems earlier, reducing their impact. Simple, working builds allow new team members to ramp up more quickly on a project: once they understand how one piece of the system is constructed, they can apply that knowledge to the entire system and move on to doing useful work. If releases, the points where code is made available outside the development team, are done using the same build system that developers use in daily life, there will be fewer surprises during releases as the “release” build process will be well-understood from development.

Builds Have Needs, Too

In 1947, Abraham Maslow described a hierarchy of needs for a person's physical and mental well-being on the premise that all the items at the lowest level of the hierarchy must be met before a person will be able to focus usefully on higher-level needs. Maslow's hierarchy begins with a set of needs that, without which, you do not have a person (for long)—physiological needs like “breathing,” “food,” and “water.” At the peak, there are extremely high-level needs that are about being a happy and enlightened person—“creativity,” “morality,” “curiosity,” and so on.

A three-tier pyramid. At the bottom: Automatable. Repeatable. Standardized.
Extensible. Understood. In the middle tier: Simple. Fast. Unit tests. Part of
the project. Environment independent. At the top: Metrics. Parallel builds.
Acceptance tests. Product caching. IDE
integration.

Builds, and software engineering as a whole, can be described the same way: at the top of the hierarchy is a working system that solves a problem, and at the bottom are the things you need to have software at all. If you don't meet needs at a given level, you will eventually be forced to stop what you're doing at a higher level and face them.

Before a build is a build, there are five key needs to meet:

If you have these five things, you have a working build. The next step is to make it comfortable. Comfortable builds can be used daily for development work, demonstrations, and tests as well as during releases; builds that are used constantly don't get a chance to “rust” as developers ignore them until a release or a demo and don’t hide surprises for launch day.

Finally, there are “chrome” features that take a build from effective to excellent. These vary widely from project to project and from organization to organization. Here are some common chrome needs:

What Doesn’t Work

Builds, like any other part of software development, have antipatterns—recurring techniques for solving a problem that introduce more problems.

What Does Work

Similarly, there are patterns—solutions that recur naturally and can be applied to many problems.

Pick A Tool, Any Tool

Nothing here is new. The value of build systems has been discussed in great detail elsewhere. Much of the accumulated build wisdom of the software industry has already been incorporated to one degree or another into build tools. What matters is that you pick one, then use it with the discipline needed to get repeatable results without thinking.