15 GitLab Runner Optimization Tips

15 GitLab Runner optimization tips: concurrent, request_concurrency, S3 cache, fastzip, pull policy, shallow clone, needs, rules, autoscaling and metrics.

Dec 24, 2025 - 16:24
Nov 10, 2026 - 04:03
 0  212
15 GitLab Runner Optimization Tips

To speed up GitLab Runner, tune concurrent and request_concurrency, use a shared S3 or GCS cache keyed on lock files, enable fastzip, pull images only when missing, clone shallowly, run jobs as a DAG with needs, skip unchanged work with rules:changes, cancel redundant pipelines, autoscale with the Docker Autoscaler executor and monitor runner metrics.

Slow pipelines usually come from three places: jobs waiting for a free runner, time spent fetching code, images and caches, and work that did not need to run at all. The 15 tips below cover all three, split between runner configuration (config.toml) and pipeline configuration (.gitlab-ci.yml). Every key and value was checked against the GitLab Runner and GitLab CI/CD documentation in November 2026. For complete pipeline files, see our 15 GitLab CI/CD pipeline examples, the hub for GitLab posts on this blog.

Key takeaways

  • Measure first: queue time, job duration and cache hit rate tell you whether the problem is capacity, I/O or wasted work.
  • concurrent is the global job limit for the whole runner process; limit caps each registered runner; request_concurrency controls how many job requests it sends at once.
  • A distributed cache in object storage is required once jobs run on more than one machine or on autoscaled instances.
  • The Docker Machine executor is deprecated. New autoscaling setups should use the Docker Autoscaler or Instance executor with a fleeting plugin, or the Kubernetes executor.
  • Pipeline changes (needs, rules:changes, interruptible, parallel) often save more time than bigger machines.

The 15 tips at a glance

#TipWhereFixes
1Size concurrent and limitconfig.tomlQueue time, overloaded hosts
2Raise request_concurrency on busy runnersconfig.tomlSlow job pickup
3Use a distributed cacheconfig.tomlCache misses across machines
4Key caches on lock files, set cache policy.gitlab-ci.ymlStale or slow caches
5Enable fastzip and tune compressionCI variablesSlow cache and artifact archiving
6Use if-not-present pull policy where safeconfig.tomlRepeated image pulls
7Use the Dependency Proxy or a registry mirror.gitlab-ci.ymlDocker Hub rate limits, slow pulls
8Clone shallowly and fetch instead of cloneCI variablesSlow Git checkout
9Build a DAG with needs.gitlab-ci.ymlStages waiting on unrelated jobs
10Skip work with rules:changes.gitlab-ci.ymlJobs that did not need to run
11Split work with parallel.gitlab-ci.ymlLong test jobs
12Cancel redundant pipelines with interruptible.gitlab-ci.ymlWasted runner minutes
13Autoscale with Docker Autoscalerconfig.tomlFixed capacity, idle cost
14Set Kubernetes executor requests and limitsconfig.tomlNoisy neighbours, evictions
15Monitor runners and expire artifactsconfig.toml, .gitlab-ci.ymlBlind capacity planning, storage bloat

Runner capacity

1. Size concurrent and limit to the host

concurrent, at the top of config.toml, limits how many jobs can run at the same time across all runners registered in that process. Each [[runners]] entry can also set limit to cap jobs for that runner. If concurrent is left at a low value, jobs queue even when the machine is idle. If it is too high, jobs fight over CPU, memory and disk.

concurrent = 8
check_interval = 3

[[runners]]
  name = "docker-build-01"
  url = "https://gitlab.example.com"
  executor = "docker"
  limit = 6
  request_concurrency = 4

Start with roughly one concurrent job per two vCPUs for build-heavy work, then adjust from metrics. check_interval sets the seconds between checks for new jobs; lowering it shortens pickup delay at the cost of more requests to GitLab.

2. Raise request_concurrency on busy runners

request_concurrency limits how many concurrent requests for new jobs a runner sends to GitLab. The default is 1, which means a busy runner asks for one job at a time. On a runner that serves many projects, raising it lets the runner fill free slots faster. Keep it below limit, and watch GitLab server load if you run a self-managed instance with many runners.

Cache and artifacts

3. Use a distributed cache

By default the cache lives on the runner host. Once jobs run on several machines, autoscaled VMs or Kubernetes pods, a job often lands on a host without the cache. A distributed cache stores archives in S3, Google Cloud Storage or Azure Blob Storage so every runner can read them. Shared = true lets runners share the same cache path.

[[runners]]
  [runners.cache]
    Type = "s3"
    Path = "gitlab-runner"
    Shared = true
    [runners.cache.s3]
      ServerAddress = "s3.amazonaws.com"
      BucketName = "example-runner-cache"
      BucketLocation = "ap-south-1"
      AuthenticationType = "iam"

Use an IAM role (or workload identity on GCP and Azure) instead of static access keys, and add a lifecycle rule on the bucket to delete old cache objects.

4. Key caches on lock files and set the cache policy

A cache keyed on the branch name is often stale or missing. Key it on the dependency lock file instead, so it changes only when dependencies change. Then set policy: pull on jobs that only read the cache, so they do not spend time uploading it again.

.node_cache: &node_cache
  key:
    files:
      - package-lock.json
  paths:
    - .npm/

install:
  stage: build
  script:
    - npm ci --cache .npm --prefer-offline
  cache:
    <<: *node_cache
    policy: pull-push

test:
  stage: test
  script:
    - npm ci --cache .npm --prefer-offline
    - npm test
  cache:
    <<: *node_cache
    policy: pull

Cache dependency downloads (such as .npm/ or ~/.m2), not build outputs you need in later jobs. Pass build outputs with artifacts, which are guaranteed; caches are best effort.

5. Enable fastzip and tune compression

The runner's default archiver can be slow with many small files, such as node_modules. The FF_USE_FASTZIP feature flag switches to a faster archiver that uses multiple cores. With fastzip enabled, you can set CACHE_COMPRESSION_LEVEL and ARTIFACT_COMPRESSION_LEVEL to fastest, fast, default, slow or slowest.

variables:
  FF_USE_FASTZIP: "true"
  CACHE_COMPRESSION_LEVEL: "fast"
  ARTIFACT_COMPRESSION_LEVEL: "fast"

Choose fastest when the cache store is close and bandwidth is cheap, and a higher level when uploads cross regions. Measure the cache upload and download times in job logs before and after.

Images and Git

6. Use the if-not-present pull policy where it is safe

The Docker executor's pull_policy accepts always (the default), if-not-present and never. if-not-present reuses an image already on the host, which saves time for large build images. You can list more than one policy so the runner falls back if the first fails.

[runners.docker]
  image = "alpine:3.22"
  pull_policy = ["if-not-present", "always"]
  allowed_pull_policies = ["always", "if-not-present"]

Use if-not-present only on runners dedicated to trusted projects. On shared runners, a job could use a private image that another project pulled earlier, so GitLab recommends always there. Pin image tags or digests so "already present" means the version you expect.

7. Use the Dependency Proxy or a registry mirror

Pulling base images from Docker Hub on every job is slow and can hit Docker Hub pull rate limits. GitLab's Dependency Proxy caches Docker Hub images at the group level. Reference it with the predefined variable:

build:
  image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/python:3.13-slim
  script:
    - pip install -r requirements.txt

For self-managed runners you can also configure a registry mirror in the Docker daemon on the host. Our container registry guide covers other mirror options.

8. Clone shallowly and fetch instead of clone

Large repositories spend minutes in checkout. GIT_DEPTH sets a shallow clone depth (new projects default to 20 in project CI/CD settings), and GIT_STRATEGY: fetch reuses the existing working copy on the runner instead of cloning fresh. Jobs that do not need the source at all, such as a deploy job that only uses artifacts, can set GIT_STRATEGY: none.

variables:
  GIT_DEPTH: "10"
  GIT_STRATEGY: fetch

deploy:
  stage: deploy
  variables:
    GIT_STRATEGY: none
  script:
    - ./deploy.sh

Tools that compute versions from Git history, or that diff against the default branch, may need a deeper clone. Set GIT_DEPTH per job where that applies.

Pipeline structure

9. Build a DAG with needs

Stages run in order: every job in test waits for every job in build. needs lets a job start as soon as the specific jobs it depends on finish, turning the pipeline into a directed acyclic graph. Independent services in a monorepo no longer wait for each other.

build_api:
  stage: build
  script: make -C api build

build_web:
  stage: build
  script: make -C web build

test_api:
  stage: test
  needs: [build_api]
  script: make -C api test

test_web:
  stage: test
  needs: [build_web]
  script: make -C web test

10. Skip work with rules:changes

Running every job on every commit wastes runner time. rules:changes runs a job only when matching files changed. Combine it with needs: optional so downstream jobs still work when an upstream job was skipped.

test_web:
  stage: test
  script: make -C web test
  rules:
    - changes:
        paths:
          - web/**/*
        compare_to: refs/heads/main

compare_to makes the comparison against the default branch, which avoids surprises on new branches and in merge request pipelines.

11. Split long jobs with parallel

parallel: N runs N copies of a job, each with CI_NODE_INDEX and CI_NODE_TOTAL set so your test runner can pick a slice. parallel:matrix runs one job per combination of variables, such as versions or targets.

rspec:
  stage: test
  parallel: 4
  script:
    - bundle exec rspec $(./ci/split-tests.sh $CI_NODE_INDEX $CI_NODE_TOTAL)

build_images:
  stage: build
  parallel:
    matrix:
      - ARCH: [amd64, arm64]
  tags: ["$ARCH"]
  script:
    - docker build --platform linux/$ARCH -t app:$ARCH .

Parallel jobs only help if there are free runner slots, so revisit tip 1 when you add them.

12. Cancel redundant pipelines with interruptible

When a developer pushes three commits in a row, the first two pipelines are usually useless. Mark safe jobs as interruptible: true and enable "Auto-cancel redundant pipelines" in the project's CI/CD settings. GitLab then cancels older running pipelines on the same branch when a new one starts. Leave deploy jobs non-interruptible so a deployment is never cut off halfway.

default:
  interruptible: true

deploy_prod:
  stage: deploy
  interruptible: false
  script: ./deploy.sh

The workflow:auto_cancel keyword gives finer control in newer GitLab versions; check the CI/CD YAML reference for your version.

Scaling and operations

13. Autoscale with the Docker Autoscaler executor

Fixed runner hosts are either too small at peak or idle at night. GitLab deprecated the Docker Machine executor in GitLab 17.5, with removal planned for GitLab 20.0. Its replacement is the GitLab Runner Autoscaler: the Docker Autoscaler and Instance executors, with fleeting plugins that manage instance groups on AWS, Google Cloud and Azure.

[[runners]]
  name = "autoscaled-docker"
  executor = "docker-autoscaler"
  [runners.docker]
    image = "alpine:3.22"
  [runners.autoscaler]
    plugin = "aws"
    capacity_per_instance = 2
    max_use_count = 20
    max_instances = 10
    [runners.autoscaler.plugin_config]
      name = "gitlab-runner-asg"
      region = "ap-south-1"
    [runners.autoscaler.connector_config]
      username = "ec2-user"
      use_external_addr = false
    [[runners.autoscaler.policy]]
      idle_count = 2
      idle_time = "20m0s"

Install the plugin with gitlab-runner fleeting install after setting plugin, then check the exact plugin name and options in the fleeting plugin's README for your cloud. Use an idle policy that keeps a small warm pool during working hours and scales down outside them. If your jobs already run on Kubernetes, the Kubernetes executor plus cluster autoscaling is the simpler path.

14. Set Kubernetes executor requests and limits

With the Kubernetes executor, each job runs as a pod. Without requests, the scheduler packs too many build pods onto one node and they slow each other down or get evicted. Set default requests and limits, and allow projects to raise them within a ceiling.

[runners.kubernetes]
  namespace = "gitlab-runner"
  cpu_request = "1"
  memory_request = "2Gi"
  cpu_limit = "2"
  memory_limit = "4Gi"
  cpu_request_overwrite_max_allowed = "4"
  memory_request_overwrite_max_allowed = "8Gi"

A job can then ask for more with KUBERNETES_CPU_REQUEST and KUBERNETES_MEMORY_REQUEST variables, up to the allowed maximum. Run build pods on a dedicated node pool so they do not compete with production workloads. For more on Kubernetes resources, see Kubernetes scenario questions.

15. Monitor runners and expire artifacts

GitLab Runner exposes Prometheus metrics when you set listen_address in config.toml. Port 9252 is the conventional choice. Metrics include running jobs, job duration histograms, request concurrency and errors, which together show whether you need more capacity or better pipelines.

listen_address = ":9252"

Also stop artifacts from filling storage. Set expire_in on artifacts that are only needed within a pipeline:

build:
  script: make build
  artifacts:
    paths: [dist/]
    expire_in: 1 week

Our list of CI/CD monitoring tools shows how to graph these metrics next to pipeline success rates.

Find your bottleneck first

SymptomLikely causeTips to apply
Jobs sit in "pending" for minutesNot enough runner slots1, 2, 13
"Restoring cache" or "Pulling image" takes most of the jobI/O and network3, 4, 5, 6, 7
Checkout is slowLarge repository history8
Pipeline is long but runners are idleStage ordering, serial tests9, 11
Many pipelines for small changesRunning everything every time10, 12
Random slow or failed jobs on KubernetesResource contention14

For failures rather than slowness, see 12 common CI/CD pipeline failures and fixes. To prepare for interviews on this topic, try our GitLab CI/CD interview questions.

Learn GitLab CI/CD hands-on

Our GitLab training covers pipelines, runners, caching, environments and security scanning with hands-on labs. Classes run in Kharadi, Pune, and live online across India.

Frequently asked questions

How do I make GitLab Runner faster?

Find the bottleneck first. Add runner capacity if jobs wait in pending, use a distributed cache and fastzip if cache and image pulls dominate, use shallow clones for slow checkout, and use needs, rules:changes and parallel to cut wasted or serial work.

What is the difference between concurrent and limit in GitLab Runner?

concurrent is a global setting that limits jobs across all runners in one runner process. limit is set per registered runner and caps jobs for that runner only. The lower effective value wins.

What does request_concurrency do?

It limits how many concurrent requests for new jobs a runner sends to GitLab. The default is 1. Raising it on a busy runner helps it fill free job slots faster.

Is the Docker Machine executor deprecated?

Yes. GitLab announced its deprecation in GitLab 17.5, with removal planned for GitLab 20.0. GitLab recommends the GitLab Runner Autoscaler, using the Docker Autoscaler or Instance executor with fleeting plugins for AWS, Google Cloud and Azure.

Should I use cache or artifacts to pass files between jobs?

Use artifacts for build outputs that later jobs need, because artifacts are guaranteed to be available. Use cache for dependency downloads that speed up jobs but can be rebuilt if missing.

Is if-not-present pull policy safe?

It is safe on runners dedicated to trusted projects. On shared runners, a job could reuse a private image pulled by another project, so GitLab recommends the always policy there.

How do I monitor GitLab Runner?

Set listen_address in config.toml to expose Prometheus metrics, commonly on port 9252, then scrape them with Prometheus. Track running jobs, job duration and errors, and graph them in Grafana.

How do I stop old pipelines from wasting runner time?

Mark safe jobs as interruptible and turn on auto-cancel redundant pipelines in project settings. GitLab then cancels older pipelines on the same branch when a new commit arrives. Keep deploy jobs non-interruptible.

Official references

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Angry Angry 0
Sad Sad 0
Wow Wow 0
Mridul I am a passionate technology enthusiast with a strong focus on DevOps, Cloud Computing, and Cybersecurity. Through my blogs at DevOps Training Institute, I aim to simplify complex concepts and share practical insights for learners and professionals. My goal is to empower readers with knowledge, hands-on tips, and industry best practices to stay ahead in the ever-evolving world of DevOps.