Continuous integration is critical part of software delivery, it contains build, test and package stages to ship your product. So, it is important that, it should be quick otherwise the product delivery may get late and it will affect the productivity.
How fast is fast? In 1999 Kent Beck, in the landmark book Extreme Programming Explained, set the bar to a maximum of 10 minutes. Everything — from building to deployment — should occur in that timeframe. Is your build up to the pace? Well, again it will be based on your product, but you can check following recommendation, to verify your build to make it faster.
1. Use powerful machines
A CI/CD pipeline is like a train: it takes our code and delivers a package to deploy. It will contain more steps based on your environment, and each steps run differently with different resources. Adding additional resources will help to complete some steps quicker. You may find processor-based tasks, like building packages, Docker images, or performing end-to-end tests, run faster.
When we speak about increasing the resource, it is important to understand where we are running the builds. In case if you are running your build server on-promise it will be hard to do it, as there is lot of processes included. But same time if we keep out build server on cloud, it will be flexible for us, to create the build agent on-demand or we can spin-up and turn off when ever we needed. Also, you have option to increase the resource on-your requirement.
Enable suitable monitoring while running your builds and check how it the resources consumption like CPU, memory etc and which stage/step consuming more. So, you can change increase or assign high resource instances for the particular build step.
2. Parallelization
Parallelization is the fastest way of improving your build time. As not all the build required to wait to complete previous builds, it can run independently, if you are kept on queue, it is right time to rethink about, as parallel running will help to complete the build faster. This may be subject to agent availability; your CI system should support the parallel builds and allow to run on multiple agents. Otherwise revalidate your CI system for quick product deliverable.
You can use cloud-based agent or containers to run your builds, so it will reduce the cost and time, also you can manage automatically.
3. Use artifacts and cache
Cache and artifacts are two types of storage features that can dramatically speed up your build time.
The cache is meant as temporary storage. You can enable cache to store the data for some time or long time. Why? The cache speeds up the build time because you don’t have to download files from the Internet on every run. Since old files are automatically deleted as needed, its primary purpose is to keep dependencies such as the node_modules folder or intermediate build files.
Artifacts, on the other hand, are like S3 buckets. They are meant as persistent or semi-persistent storage to keep data and executables for a longer time. While free during the current beta period, artifacts will eventually have a per-usage charge.
There are three categories of artifacts, which allows you to choose an optimal retention policy:
- Job artifacts: used to collect job-specific data such as debug info, logs, and UI screenshots.
- Workflow artifacts: suitable for passing data and builds from one job to another, permitting you to build once and test many times.
- Project artifacts: used for storing final deliverables.
4. Optimize tests
In most of the builds, testing is one of long running step, as it may contain hundreds or thousands of tests, so this is where most of the CI/CD time is consumed. Sametime, tests are mostly no resource consuming stage, so adding resource wont helpful in this case.
How to improve this? the solution is debug and optimize tests. For that, we must first identify slow or time-consuming tests, in this case you can either separate complex test or you can make it done after some basic validation. The easiest way of finding the bottlenecks in your pipeline is to configure test reports, which gives you aggregated statistics about your jobs across all runs.
Identifying slow jobs is only the first step of the plan, however. To remove them, you should:
- Check the test is valid or not or can be skipped/dropped.
- Run a profiler on the job to see what’s causing the slowdown.
- Once the problem is identified, optimize the job to remove the bottleneck.
5. Change CI/CD solution
If, despite all your efforts, the builds are still slow, it may not be your fault. It may be that you’re using the wrong tool.
There are many CI/CD solutions to choose from, with different features and use cases. Some, like Jenkins, require you to set up, configure, and manage hardware. You are responsible for the reliability of the system and scaling it up as your project grows. While this allows you a great degree of control, you might not have the time or inclination to micromanage everything in some situations.
There are more CI/CD solutions, like CircleCI, TravisCI, Semaphore, etc, this also offers cloud-based solutions. For this, there is no management required; you can just sign-up and get to work immediately. These solutions are the fastest and most flexible, as they can scale up automatically as needed.
Keep builds fast
Speeding up your CI/CD build is never wasted effort. As long you save time on build you can deliver product faster and you can increase the release cycle also. Keeping your CI/CD pipeline defined and fast will ensure that your development team reaches its maximum potential.
You can check about GitOps https://foxutech.com/lets-understand-about-gitops/