Continuous integration (CI) and continuous deployment (CD) run at the center of agile development. DevOps teams around the world use CI/CD to make software programming more efficient. Read on to learn how to accelerate a CI/CD pipeline and make it more flexible.
CI/CD pipelines – meaning and problems
Pipelines are the backbone of any CI/CD process as they are responsible for executing a series of tasks to ensure code is built, tested and deployed successfully.
The pipeline is a set of stages that runs in sequence, with each stage being a script or an executable that performs one task. The pipeline can also be configured to execute tests for code quality, run static analysis tools on the code, or deploy the code to production.
The most popular workflow for pipelines is a git-flow branching strategy, which divides the development process into four major branches: main, develop, feature and release branches.
We started one of our projects with a classic git workflow approach that utilized three types of branches:
- One main branch – reflecting the current version in the app store
- One develop branch – a collection of approved and tested feature branches, merged into the main branch on release
- Multiple feature branches – merged into development when a feature was completed and tested
This led to a number of problems. The develop branch was always behind what was actually implemented, due to pending tests. The process didn’t feel right at all – the flow was confusing and ambiguous. We needed to find another solution, one that would speed up the process and allow for more flexibility in the way developers worked on a project.
Moving to trunk-based development
Trunk-based development is a software development process that is based on the idea of having one main branch in which all changes are made. This branch is called the trunk, and is the place where all new features and bug fixes are added.
In October 2021 we transitioned from the standard multi-branch git-flow to trunk-based development. The new approach made us abandon long-lived feature branches, which, in the past, resulted in conflicts and infrequent test version deployments.
Pipelines – white-label YAML config file
The Azure DevOps YAML config file is a configuration file that can be used to customize the build process on the development machine. It is typically used to specify the build steps, plugins and environment variables to use when building an application.
We used separate pipelines for each app flavor (a concrete implementation of a white-label app). A single YAML file configured all flavors – and the flavor-specific differences were controlled by pipeline variables.
For example, in the case of Android apps, the command:
| assemble$(product_flavor)ApiDevDebug |
will build any of the available flavors – the value of product_flavor is set in the variables section of each pipeline.
The problem
Workflow changes caused unintended side effects in the pipelines. Before the changes, we used pull requests to schedule new builds, but, once we were left with a single branch, a push to the main branch triggered new builds every time new code was pushed. This resulted in the building of all five flavors of the app in both test environments (dev and UAT), for 10 builds in total (20 on both platforms!).
We didn’t want the pipeline to build all flavors as it was redundant to test all of the apps every time. Our goal was to find a solution that would allow the team to:
- Push work-in-progress changes without triggering the pipeline
- Build just the main flavor of the app
- Skip unit tests (e.g. when making UI changes)
- Trigger a release pipeline
Commit flags to the rescue
We decided to control pipelines by using flags in commit messages. The pipeline analyzes the last commit message and checks if flags are present, and if so, triggers or skips relevant actions.
For example:
–skip-dev-build – skips dev environment build
–skip-test-build – skips UAT environment build
–build-all – builds all app flavors
–skip-unit-tests – skips unit tests
–release – triggers the release pipeline that deploys the app to internal tracks in app stores
Flags can be combined. So, a commit with flags:
| “Commit message –build-all –skip-unit-tests –skip-test-build” |
will skip unit tests, output a development build of all white label product flavors and skip the UAT builds.
Evaluation stage
We added a new stage in the YAML config file called Evaluation.
| – stage: Evaluation jobs: – job: EvaluateDeploymentConditions displayName: ‘Evaluate deployment conditions’ |
Then we checked each flag in a separate task. When a flag was present, the task set an associated pipeline variable to true:
| – task: Bash@3 name: ‘CheckBuildAllFlag’ displayName: ‘–build-all’ condition: contains(variables[‘Build.SourceVersionMessage’], variables.BuildAllFlag) inputs: targetType: ‘inline’ script: | echo “##vso[task.setvariable variable=buildAllApps;isOutput=true;]true” echo ‘Found BuildAllFlag.’ |
Further stages required specific conditions to be met in order to run:
| – stage: DistributionDev dependsOn: [Evaluation, Test] displayName: ‘Distribution – Dev’ condition: ne(dependencies.Evaluation.outputs[‘EvaluateDeploymentConditions.SkipDevBuildFlag Evaluation.skipDevBuild’], ‘true’) |
So, the DistributionDev stage ran only when the –skip-dev-build flag was not present and ran in any other case.
Results
Commit flags have enabled us to stay with the single branch approach and not litter the deployment pipelines by releasing new versions of the app only when they were actually needed.
On top of that, it is now possible to check a CI history just by looking at the version control log, e.g. each release build now has a –release flag next to it. Thus, it is now very easy to go through the release history or check the last date of white-label apps distribution. However, one can argue that the Git log is now cluttered by all of the flags.
CI/CD pipelines and commit flags
Commit flags are a great way to save time and resources for developers, testers and product managers. Beyond making it easier to deploy relevant builds, they also increase a team’s ability to maintain the code through the use of a single generic configuration file.
Developers can implement custom behavior and control pipeline stages on the fly. It is possible to avoid redundancy and resources usage by skipping unnecessary steps. A white-label oriented pipeline can produce only the “main” flavor and versions that are relevant only at a given time. Although the version control log may appear cluttered by the flags, developers can see the release history and other important CI information directly in the IDE (Integrated development environment).
If you want to dive deeper into CI/CD pipelines and learn how to make your software deployment process more efficient while saving your team’s time and your company’s resources, get in touch by using the contact form.
About the authorAdrian Zając
Senior Software Engineer
With a love for solving problems and an innate need to build and create, Adrian Zając is constantly looking for challenges where his outside-the-box thinking might be valuable. In his 4 years of experience as an Android Developer, he has worked on a variety of projects and learned a lot of skills of the trade. Aside from mobile development, he loves making music, creating games, ice skating, hiking and cooking.
