A short story about Pre-tested Integration

Continuous Integration and Code Review are strongly correlated with success. Many use Pull Requests for code review, but for co-located teams this can be an obstacle for CI. Is there a better way?

There are three developers on the (fictitious) team: Annika, Boris and Carol. Annika is a recent hire, fresh from university, Boris is the team lead, and Carol has been around the longest. Each of them is working on a different task. They all synchronized their work with their shared master branch when they arrived at the office today, and now it’s approaching morning coffee time. They have all made some changes in the code which they’d like to share with the rest of the team.

Annika is working on a local branch called ‘red’. She checks it’s up-to-date with master and pushes it to a remote branch named ‘ready/red’. It’s similar for Boris and Carol. They are on blue and orange branches respectively and push their changes to ready/blue and ready/orange.

the three

The Build Server is set up so that it detects new branches on the Version Control Server that follow a naming convention. Any branch beginning with ‘ready/’ is scheduled for integration, and only one of these integration builds runs at a time. The Build Server delegates builds to one or more agents,and since the ready-job agent is idle, it picks up the ‘ready/red’ change straight away and leaves the other two ready-branch builds in the queue.

build servers

The build job has several steps. First, the agent merges the ready-branch into a local copy of the master branch. Annika’s changes get a simple fast-forward. The agent performs a full build, static analysis, code style check, and unit test. Everything goes well, so the agent pushes the merge result up to the Version Control Server and posts a message on the team message board.

Things start out similarly for Boris’ ready/blue branch. The build agent takes a copy of master from the git server and merges in the ready-branch. This isn’t a fast-forward merge, since there is a new commit in master for the ‘red’ changes, but it’s still ok. So long as the agent can do the merge without finding any conflicts the build can continue.

The agent then proceeds to the next build steps. Unfortunately, Boris hadn’t noticed that one of his changes caused a test failure. The team has previously agreed that the code in master should always pass the tests, so this means Boris’ changes shouldn’t be shared. The build agent sends a message to Boris telling him about the failed tests, discards its merged branch, and moves on. Carol’s ready/orange branch is up next. The build agent starts again with a fresh copy of the latest master taken from the git server. Carol’s changes also merge without difficulty and this time both build and tests pass. The build agent pushes the merge commit to the server and notifies the team.

build servers

Boris and Carol are having a cup of coffee while they wait for the build server to integrate their changes. Annika is chatting with the Product Owner about the new feature she plans to work on next, ‘cyan’. When they get back to their desks they see the messages from the build server.

Annika is happy to see her changes integrated successfully. She fetches the latest master from the remote git server. She’s now completed the work on the ‘red’ task, and her changes should undergo a code review. She marks the ‘red’ task as finished in the issue tracker and adds an agenda item to the team’s next scheduled code review meeting, which is later that week. Annika selects a new task to work on and checks out a local branch from master called ‘cyan’. Boris sees the message about his failed tests and realizes immediately what he missed. He’s a little embarrassed about his mistake, but happy his teammates are not affected. They may not even notice what’s happened. Boris takes the opportunity to merge the latest changes from master into his ‘blue’ branch. He is quickly able to address the problem with the tests and pushes an update to ready/blue. The build agent gets to work straight away.

Carol is not finished with the ‘orange’ task, but is happy to see her initial changes integrated successfully. She fetches master and merges it into ‘orange’ before continuing work there. She’s noticed a design change that would make her task easier. She plans the refactoring in steps so she can push small changes frequently as she completes the re-design. Sharing her changes with the team often will make it easier for everyone to avoid costly merges. Later that week, in the code review meeting, the team looks at Annika’s changes for the ‘red’ task. It represents a couple of days’ work. The code review tool presents a summary of all the commits involved and they discuss all the changes in the development of the ‘red’ feature.

Unfortunately, Boris and Carol are not happy with a part of the design Annika has made and the code formatting needs improving in places. The outcome of the meeting is that they agree to pair program with Annika on a refactoring of the design, and encourage her to initiate informal design discussions more often during development. The idea is that the more experienced developers, Boris and Carol, should help Annika to learn better design skills. The team finds the code-formatting issues a bit annoying since this kind of detail shouldn’t be in focus for a code review meeting. They create a task to improve the code-style checker in the pre-tested integration build to catch any similar code formatting problem in future.

Commentary

This development process is working really well for Annika, Boris and Carol, and pre-tested integration is a small but important piece. They are not using pull requests, but they have checks on what code is allowed into the master branch, and they have good code-review culture. Integration to master happens at a faster cadence than the flow of work-items. That’s important. Integration is less painful the more often you do it and you might not want to break your work items down to the same small granularity that would be best for code changes. You also don’t necessarily want to delay your integration by waiting for a teammate to review your pull request.

Strictly speaking, this process is not Trunk-Based development since there are more branches involved than just trunk, but so long as the integration is frequent in practice it’s indistinguishable. The benefit of this over Trunk-Based development is, of course, that Boris or any other developer can’t unwittingly break master for the rest of the team.

If you’re using Jenkins you can easily automate the integration process with our Pretested Integration plugin. It’s not difficult to implement this functionality yourself for other build servers. Whichever approach your team chooses, I recommend you settle on a process that results in frequent integration together with collaborative and constructive code-reviews.

See how we can help you with Continuous Delivery

Published: Feb 23, 2018

Updated: Jun 1, 2021

CI/CD