15 GitLab Runner Optimization Tips
15 GitLab Runner optimization tips: concurrent, request_concurrency, S3 cache, fastzip, pull policy, shallow clone, needs, rules, autoscaling and metrics.
To speed up GitLab Runner, tune concurrent and request_concurrency, use a shared S3 or GCS cache keyed on lock files, enable fastzip, pull images only when missing, clone shallowly, run jobs as a DAG with needs, skip unchanged work with rules:changes, cancel redundant pipelines, autoscale with the Docker Autoscaler executor and monitor runner metrics.
Slow pipelines usually come from three places: jobs waiting for a free runner, time spent fetching code, images and caches, and work that did not need to run at all. The 15 tips below cover all three, split between runner configuration (config.toml) and pipeline configuration (.gitlab-ci.yml). Every key and value was checked against the GitLab Runner and GitLab CI/CD documentation in November 2026. For complete pipeline files, see our 15 GitLab CI/CD pipeline examples, the hub for GitLab posts on this blog.
Key takeaways
- Measure first: queue time, job duration and cache hit rate tell you whether the problem is capacity, I/O or wasted work.
concurrentis the global job limit for the whole runner process;limitcaps each registered runner;request_concurrencycontrols how many job requests it sends at once.- A distributed cache in object storage is required once jobs run on more than one machine or on autoscaled instances.
- The Docker Machine executor is deprecated. New autoscaling setups should use the Docker Autoscaler or Instance executor with a fleeting plugin, or the Kubernetes executor.
- Pipeline changes (
needs,rules:changes,interruptible,parallel) often save more time than bigger machines.
The 15 tips at a glance
| # | Tip | Where | Fixes |
|---|---|---|---|
| 1 | Size concurrent and limit | config.toml | Queue time, overloaded hosts |
| 2 | Raise request_concurrency on busy runners | config.toml | Slow job pickup |
| 3 | Use a distributed cache | config.toml | Cache misses across machines |
| 4 | Key caches on lock files, set cache policy | .gitlab-ci.yml | Stale or slow caches |
| 5 | Enable fastzip and tune compression | CI variables | Slow cache and artifact archiving |
| 6 | Use if-not-present pull policy where safe | config.toml | Repeated image pulls |
| 7 | Use the Dependency Proxy or a registry mirror | .gitlab-ci.yml | Docker Hub rate limits, slow pulls |
| 8 | Clone shallowly and fetch instead of clone | CI variables | Slow Git checkout |
| 9 | Build a DAG with needs | .gitlab-ci.yml | Stages waiting on unrelated jobs |
| 10 | Skip work with rules:changes | .gitlab-ci.yml | Jobs that did not need to run |
| 11 | Split work with parallel | .gitlab-ci.yml | Long test jobs |
| 12 | Cancel redundant pipelines with interruptible | .gitlab-ci.yml | Wasted runner minutes |
| 13 | Autoscale with Docker Autoscaler | config.toml | Fixed capacity, idle cost |
| 14 | Set Kubernetes executor requests and limits | config.toml | Noisy neighbours, evictions |
| 15 | Monitor runners and expire artifacts | config.toml, .gitlab-ci.yml | Blind capacity planning, storage bloat |
Runner capacity
1. Size concurrent and limit to the host
concurrent, at the top of config.toml, limits how many jobs can run at the same time across all runners registered in that process. Each [[runners]] entry can also set limit to cap jobs for that runner. If concurrent is left at a low value, jobs queue even when the machine is idle. If it is too high, jobs fight over CPU, memory and disk.
concurrent = 8
check_interval = 3
[[runners]]
name = "docker-build-01"
url = "https://gitlab.example.com"
executor = "docker"
limit = 6
request_concurrency = 4
Start with roughly one concurrent job per two vCPUs for build-heavy work, then adjust from metrics. check_interval sets the seconds between checks for new jobs; lowering it shortens pickup delay at the cost of more requests to GitLab.
2. Raise request_concurrency on busy runners
request_concurrency limits how many concurrent requests for new jobs a runner sends to GitLab. The default is 1, which means a busy runner asks for one job at a time. On a runner that serves many projects, raising it lets the runner fill free slots faster. Keep it below limit, and watch GitLab server load if you run a self-managed instance with many runners.
Cache and artifacts
3. Use a distributed cache
By default the cache lives on the runner host. Once jobs run on several machines, autoscaled VMs or Kubernetes pods, a job often lands on a host without the cache. A distributed cache stores archives in S3, Google Cloud Storage or Azure Blob Storage so every runner can read them. Shared = true lets runners share the same cache path.
[[runners]]
[runners.cache]
Type = "s3"
Path = "gitlab-runner"
Shared = true
[runners.cache.s3]
ServerAddress = "s3.amazonaws.com"
BucketName = "example-runner-cache"
BucketLocation = "ap-south-1"
AuthenticationType = "iam"
Use an IAM role (or workload identity on GCP and Azure) instead of static access keys, and add a lifecycle rule on the bucket to delete old cache objects.
4. Key caches on lock files and set the cache policy
A cache keyed on the branch name is often stale or missing. Key it on the dependency lock file instead, so it changes only when dependencies change. Then set policy: pull on jobs that only read the cache, so they do not spend time uploading it again.
.node_cache: &node_cache
key:
files:
- package-lock.json
paths:
- .npm/
install:
stage: build
script:
- npm ci --cache .npm --prefer-offline
cache:
<<: *node_cache
policy: pull-push
test:
stage: test
script:
- npm ci --cache .npm --prefer-offline
- npm test
cache:
<<: *node_cache
policy: pull
Cache dependency downloads (such as .npm/ or ~/.m2), not build outputs you need in later jobs. Pass build outputs with artifacts, which are guaranteed; caches are best effort.
5. Enable fastzip and tune compression
The runner's default archiver can be slow with many small files, such as node_modules. The FF_USE_FASTZIP feature flag switches to a faster archiver that uses multiple cores. With fastzip enabled, you can set CACHE_COMPRESSION_LEVEL and ARTIFACT_COMPRESSION_LEVEL to fastest, fast, default, slow or slowest.
variables:
FF_USE_FASTZIP: "true"
CACHE_COMPRESSION_LEVEL: "fast"
ARTIFACT_COMPRESSION_LEVEL: "fast"
Choose fastest when the cache store is close and bandwidth is cheap, and a higher level when uploads cross regions. Measure the cache upload and download times in job logs before and after.
Images and Git
6. Use the if-not-present pull policy where it is safe
The Docker executor's pull_policy accepts always (the default), if-not-present and never. if-not-present reuses an image already on the host, which saves time for large build images. You can list more than one policy so the runner falls back if the first fails.
[runners.docker]
image = "alpine:3.22"
pull_policy = ["if-not-present", "always"]
allowed_pull_policies = ["always", "if-not-present"]
Use if-not-present only on runners dedicated to trusted projects. On shared runners, a job could use a private image that another project pulled earlier, so GitLab recommends always there. Pin image tags or digests so "already present" means the version you expect.
7. Use the Dependency Proxy or a registry mirror
Pulling base images from Docker Hub on every job is slow and can hit Docker Hub pull rate limits. GitLab's Dependency Proxy caches Docker Hub images at the group level. Reference it with the predefined variable:
build:
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/python:3.13-slim
script:
- pip install -r requirements.txt
For self-managed runners you can also configure a registry mirror in the Docker daemon on the host. Our container registry guide covers other mirror options.
8. Clone shallowly and fetch instead of clone
Large repositories spend minutes in checkout. GIT_DEPTH sets a shallow clone depth (new projects default to 20 in project CI/CD settings), and GIT_STRATEGY: fetch reuses the existing working copy on the runner instead of cloning fresh. Jobs that do not need the source at all, such as a deploy job that only uses artifacts, can set GIT_STRATEGY: none.
variables:
GIT_DEPTH: "10"
GIT_STRATEGY: fetch
deploy:
stage: deploy
variables:
GIT_STRATEGY: none
script:
- ./deploy.sh
Tools that compute versions from Git history, or that diff against the default branch, may need a deeper clone. Set GIT_DEPTH per job where that applies.
Pipeline structure
9. Build a DAG with needs
Stages run in order: every job in test waits for every job in build. needs lets a job start as soon as the specific jobs it depends on finish, turning the pipeline into a directed acyclic graph. Independent services in a monorepo no longer wait for each other.
build_api:
stage: build
script: make -C api build
build_web:
stage: build
script: make -C web build
test_api:
stage: test
needs: [build_api]
script: make -C api test
test_web:
stage: test
needs: [build_web]
script: make -C web test
10. Skip work with rules:changes
Running every job on every commit wastes runner time. rules:changes runs a job only when matching files changed. Combine it with needs: optional so downstream jobs still work when an upstream job was skipped.
test_web:
stage: test
script: make -C web test
rules:
- changes:
paths:
- web/**/*
compare_to: refs/heads/main
compare_to makes the comparison against the default branch, which avoids surprises on new branches and in merge request pipelines.
11. Split long jobs with parallel
parallel: N runs N copies of a job, each with CI_NODE_INDEX and CI_NODE_TOTAL set so your test runner can pick a slice. parallel:matrix runs one job per combination of variables, such as versions or targets.
rspec:
stage: test
parallel: 4
script:
- bundle exec rspec $(./ci/split-tests.sh $CI_NODE_INDEX $CI_NODE_TOTAL)
build_images:
stage: build
parallel:
matrix:
- ARCH: [amd64, arm64]
tags: ["$ARCH"]
script:
- docker build --platform linux/$ARCH -t app:$ARCH .
Parallel jobs only help if there are free runner slots, so revisit tip 1 when you add them.
12. Cancel redundant pipelines with interruptible
When a developer pushes three commits in a row, the first two pipelines are usually useless. Mark safe jobs as interruptible: true and enable "Auto-cancel redundant pipelines" in the project's CI/CD settings. GitLab then cancels older running pipelines on the same branch when a new one starts. Leave deploy jobs non-interruptible so a deployment is never cut off halfway.
default:
interruptible: true
deploy_prod:
stage: deploy
interruptible: false
script: ./deploy.sh
The workflow:auto_cancel keyword gives finer control in newer GitLab versions; check the CI/CD YAML reference for your version.
Scaling and operations
13. Autoscale with the Docker Autoscaler executor
Fixed runner hosts are either too small at peak or idle at night. GitLab deprecated the Docker Machine executor in GitLab 17.5, with removal planned for GitLab 20.0. Its replacement is the GitLab Runner Autoscaler: the Docker Autoscaler and Instance executors, with fleeting plugins that manage instance groups on AWS, Google Cloud and Azure.
[[runners]]
name = "autoscaled-docker"
executor = "docker-autoscaler"
[runners.docker]
image = "alpine:3.22"
[runners.autoscaler]
plugin = "aws"
capacity_per_instance = 2
max_use_count = 20
max_instances = 10
[runners.autoscaler.plugin_config]
name = "gitlab-runner-asg"
region = "ap-south-1"
[runners.autoscaler.connector_config]
username = "ec2-user"
use_external_addr = false
[[runners.autoscaler.policy]]
idle_count = 2
idle_time = "20m0s"
Install the plugin with gitlab-runner fleeting install after setting plugin, then check the exact plugin name and options in the fleeting plugin's README for your cloud. Use an idle policy that keeps a small warm pool during working hours and scales down outside them. If your jobs already run on Kubernetes, the Kubernetes executor plus cluster autoscaling is the simpler path.
14. Set Kubernetes executor requests and limits
With the Kubernetes executor, each job runs as a pod. Without requests, the scheduler packs too many build pods onto one node and they slow each other down or get evicted. Set default requests and limits, and allow projects to raise them within a ceiling.
[runners.kubernetes]
namespace = "gitlab-runner"
cpu_request = "1"
memory_request = "2Gi"
cpu_limit = "2"
memory_limit = "4Gi"
cpu_request_overwrite_max_allowed = "4"
memory_request_overwrite_max_allowed = "8Gi"
A job can then ask for more with KUBERNETES_CPU_REQUEST and KUBERNETES_MEMORY_REQUEST variables, up to the allowed maximum. Run build pods on a dedicated node pool so they do not compete with production workloads. For more on Kubernetes resources, see Kubernetes scenario questions.
15. Monitor runners and expire artifacts
GitLab Runner exposes Prometheus metrics when you set listen_address in config.toml. Port 9252 is the conventional choice. Metrics include running jobs, job duration histograms, request concurrency and errors, which together show whether you need more capacity or better pipelines.
listen_address = ":9252"
Also stop artifacts from filling storage. Set expire_in on artifacts that are only needed within a pipeline:
build:
script: make build
artifacts:
paths: [dist/]
expire_in: 1 week
Our list of CI/CD monitoring tools shows how to graph these metrics next to pipeline success rates.
Find your bottleneck first
| Symptom | Likely cause | Tips to apply |
|---|---|---|
| Jobs sit in "pending" for minutes | Not enough runner slots | 1, 2, 13 |
| "Restoring cache" or "Pulling image" takes most of the job | I/O and network | 3, 4, 5, 6, 7 |
| Checkout is slow | Large repository history | 8 |
| Pipeline is long but runners are idle | Stage ordering, serial tests | 9, 11 |
| Many pipelines for small changes | Running everything every time | 10, 12 |
| Random slow or failed jobs on Kubernetes | Resource contention | 14 |
For failures rather than slowness, see 12 common CI/CD pipeline failures and fixes. To prepare for interviews on this topic, try our GitLab CI/CD interview questions.
Learn GitLab CI/CD hands-on
Our GitLab training covers pipelines, runners, caching, environments and security scanning with hands-on labs. Classes run in Kharadi, Pune, and live online across India.
Frequently asked questions
How do I make GitLab Runner faster?
Find the bottleneck first. Add runner capacity if jobs wait in pending, use a distributed cache and fastzip if cache and image pulls dominate, use shallow clones for slow checkout, and use needs, rules:changes and parallel to cut wasted or serial work.
What is the difference between concurrent and limit in GitLab Runner?
concurrent is a global setting that limits jobs across all runners in one runner process. limit is set per registered runner and caps jobs for that runner only. The lower effective value wins.
What does request_concurrency do?
It limits how many concurrent requests for new jobs a runner sends to GitLab. The default is 1. Raising it on a busy runner helps it fill free job slots faster.
Is the Docker Machine executor deprecated?
Yes. GitLab announced its deprecation in GitLab 17.5, with removal planned for GitLab 20.0. GitLab recommends the GitLab Runner Autoscaler, using the Docker Autoscaler or Instance executor with fleeting plugins for AWS, Google Cloud and Azure.
Should I use cache or artifacts to pass files between jobs?
Use artifacts for build outputs that later jobs need, because artifacts are guaranteed to be available. Use cache for dependency downloads that speed up jobs but can be rebuilt if missing.
Is if-not-present pull policy safe?
It is safe on runners dedicated to trusted projects. On shared runners, a job could reuse a private image pulled by another project, so GitLab recommends the always policy there.
How do I monitor GitLab Runner?
Set listen_address in config.toml to expose Prometheus metrics, commonly on port 9252, then scrape them with Prometheus. Track running jobs, job duration and errors, and graph them in Grafana.
How do I stop old pipelines from wasting runner time?
Mark safe jobs as interruptible and turn on auto-cancel redundant pipelines in project settings. GitLab then cancels older pipelines on the same branch when a new commit arrives. Keep deploy jobs non-interruptible.
Official references
- GitLab Runner: advanced configuration (config.toml)
- GitLab Runner: feature flags
- GitLab Docs: caching in GitLab CI/CD
- GitLab CI/CD YAML syntax reference
- GitLab Docs: Dependency Proxy
- GitLab Runner autoscaling
- GitLab Runner: Kubernetes executor
- GitLab Runner: monitoring
- GitLab Docs: deprecations and removals
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0