10 Best Practices to Manage Secrets in DevOps Pipelines

Ten practices for managing secrets in CI/CD: OIDC federation, one secret manager, job scoping, dynamic credentials, rotation, scanning, leak response.

Dec 23, 2025 - 18:08
Nov 10, 2026 - 16:58
 0  84
10 Best Practices to Manage Secrets in DevOps Pipelines

Managing secrets in DevOps means treating every credential as something with a lifecycle: who issues it, where it lives, how long it lasts, who can read it, how it is rotated and how a leak is handled. The strongest practices are federated short-lived credentials (OIDC), a central secret manager, least privilege per pipeline job, scanning, rotation and audited access.

This guide focuses on the delivery pipeline: CI runners, deployment jobs and the credentials they use to reach clouds, registries and clusters. Related posts cover the narrower cases: storing secrets in Git repositories, secrets in Ansible and Terraform projects and tools to manage Kubernetes Secrets. Read those for tool-specific detail; read this one to design how secrets move through your pipeline.

Key takeaways

  • The safest secret is one that does not exist. OIDC federation lets CI jobs get short-lived cloud credentials with no stored key.
  • Where a static secret is unavoidable, keep it in one secret manager, fetch it at runtime and scope it to one job and one environment.
  • CI systems mask secrets in logs, but masking is a safety net. Code that encodes or splits a value can still leak it.
  • Scan before commit, at push and in the pipeline. Assume something will slip through and have a revoke-and-rotate runbook.
  • Untrusted code (pull requests from forks, third-party actions and plugins) must never run with access to production secrets.

What counts as a secret in a pipeline

Secret typeTypical use in CI/CDBetter alternative where possible
Cloud access keysDeploying infrastructure and appsOIDC federation to a cloud role
Registry passwordsPushing and pulling imagesCloud registry auth through the same OIDC role
Kubeconfig files and tokensDeploying to clustersGitOps controller inside the cluster, or short-lived tokens
Database passwordsMigrations, integration testsDynamic credentials from a secret manager
Third-party API tokensNotifications, artifact stores, scannersNarrow-scope tokens with expiry, stored in a secret manager
Signing keysSigning images and artifactsKeyless signing (for example Sigstore cosign with OIDC) or keys held in a KMS/HSM
SSH keysDeploying to VMs, cloning private reposShort-lived SSH certificates, deploy keys per repository

1. Remove long-lived cloud keys with OIDC federation

GitHub Actions, GitLab CI/CD, Azure Pipelines and other CI systems can issue an OpenID Connect token for each job. The cloud provider trusts that issuer and exchanges the token for temporary credentials. There is no access key to store, leak or rotate.

# GitHub Actions: assume an AWS role with OIDC
permissions:
  id-token: write
  contents: read
jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production
    steps:
      - uses: actions/checkout@v4
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/deploy-production
          aws-region: ap-south-1
      - run: aws sts get-caller-identity

The security of this setup depends on the trust policy. Restrict the token's sub claim to the exact repository and branch or environment, for example repo:my-org/my-app:environment:production. A trust policy that accepts any repository in the organisation, or any branch, hands production to whoever can push a branch. GitLab provides the same capability through the id_tokens keyword, and Azure uses workload identity federation on service connections.

2. Keep remaining secrets in one secret manager

Some credentials cannot be federated: a SaaS API token, a legacy database password. Store these in a dedicated secret manager such as HashiCorp Vault, AWS Secrets Manager, Azure Key Vault or Google Secret Manager, rather than copying them into several CI systems, wiki pages and environment files.

One source of truth means one place to rotate, one access policy and one audit log. CI-native secret stores (GitHub Actions secrets, GitLab CI/CD variables, Jenkins credentials) are fine for a small number of values, but large organisations usually have the pipeline authenticate to the central manager with OIDC and fetch what it needs at runtime.

# GitLab CI: authenticate to Vault with an ID token and read a secret
deploy:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://vault.example.com
  secrets:
    DATABASE_PASSWORD:
      vault: production/db/password@ops
      token: $VAULT_ID_TOKEN
  script:
    - ./migrate.sh

The secrets:vault keyword is a GitLab Premium and Ultimate feature; on other tiers, call Vault's JWT auth method from the job script. For Vault interview-level detail, see most-asked HashiCorp Vault interview questions.

3. Scope every secret to a job and an environment

A secret available to every job in every branch is effectively available to every contributor. Narrow it down:

  • Use environment-scoped secrets (GitHub environments, GitLab protected environments and protected variables, Azure DevOps environments) so production credentials are only released to jobs deploying to production.
  • Require approvals or branch protection on those environments.
  • Give build jobs no deployment credentials at all. Separate the job that compiles untrusted code from the job that deploys.
  • Set the default GITHUB_TOKEN permissions to read-only at the organisation or repository level, and grant write per workflow.
  • Give each pipeline its own cloud role. Shared "ci-admin" roles make blast radius and audit trails much worse.

4. Prefer short-lived and dynamic credentials

A credential that expires in 15 minutes is worth little to someone who finds it in a log next week. OIDC sessions are short by design. Secret managers can go further: Vault's database secrets engine creates a new database user per request with a lease, then revokes it when the lease ends. Cloud providers offer similar short-lived tokens for their own services.

Where only static secrets are possible, set an expiry date when you create the token. Many SaaS tokens support this, and it forces a rotation conversation before the token is forgotten.

5. Rotate on a schedule and on every suspicion

Rotation limits how long a leaked secret stays useful. Automate it; manual rotation gets postponed. Managed services such as AWS Secrets Manager support scheduled rotation through Lambda functions, and Vault can rotate root credentials for its secrets engines.

Two rules make rotation safe:

  • Applications must read secrets at runtime or reload them, not bake them into images at build time.
  • Support two valid credentials during the switch (old and new) so rotation does not cause an outage.

6. Keep secrets out of code, images and logs

Secrets leak through places people forget to check:

  • Container images. A secret copied in a Dockerfile layer stays in the image history even if a later layer deletes it. Use BuildKit secret mounts instead: RUN --mount=type=secret,id=npmrc ... with docker build --secret id=npmrc,src=.npmrc.
  • Logs. CI systems mask registered secret values, but not values that are transformed (base64-encoded, split, URL-encoded). Avoid set -x in steps that handle secrets, and do not echo environment dumps.
  • Artifacts and caches. Check that .env files, kubeconfigs and Terraform state are not uploaded as build artifacts.
  • Terraform state. State can contain secrets in plain text; store it in an encrypted, access-controlled backend.
# syntax=docker/dockerfile:1
FROM node:22-slim
WORKDIR /app
COPY package*.json ./
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci
COPY . .
CMD ["node", "server.js"]

7. Scan at every stage

Layer the checks so a leaked secret is caught as early as possible:

StageExample control
Developer machinePre-commit hook running gitleaks or similar
Push to Git hostGitHub secret scanning push protection, GitLab secret push protection
PipelineSecret scanner on every merge request; image scanner checking layers
HistoryPeriodic scans of full Git history and existing images
# Scan the current repository with gitleaks
gitleaks git --verbose

# Scan a directory that is not a Git repository
gitleaks dir ./deploy

Command names above match gitleaks v8.19 and later; older versions used gitleaks detect. More pipeline security checks are listed in 14 CI/CD security best practices.

8. Isolate untrusted code from secrets

Pipelines run code from many sources: contributors, dependencies, third-party actions and plugins. Each one can read whatever the job can read.

  • On GitHub, workflows triggered by pull_request from forks do not receive repository secrets. Avoid pull_request_target workflows that check out and run the fork's code, since they run with the base repository's secrets.
  • Pin third-party actions to a full commit SHA rather than a mutable tag.
  • Review and limit Jenkins plugins and shared libraries, which run with controller-level access.
  • Use ephemeral runners for sensitive jobs, so one job cannot leave something behind that reads the next job's secrets.

9. Audit and alert on secret access

You should be able to answer "who read the production database password last week?" Secret managers log every read; cloud providers log every role assumption (for example AWS CloudTrail records AssumeRoleWithWebIdentity calls). Send these logs to your monitoring platform and alert on unusual patterns: access from an unexpected pipeline, a burst of reads, or use outside deployment windows.

Review access policies on a schedule as well. Pipelines and service accounts accumulate permissions over time and rarely lose them.

10. Have a leak response runbook

Deleting a committed secret from Git does not make it safe; it is already in clones, forks and caches. When a secret leaks:

  1. Revoke the credential immediately at the issuer.
  2. Rotate to a new value and update consumers through the secret manager.
  3. Investigate using audit logs to see whether the old credential was used and by whom.
  4. Clean up history with git filter-repo if needed, knowing this is hygiene, not remediation.
  5. Fix the cause: add the scanner rule, hook or pipeline change that would have caught it.

Practise the runbook on a dummy secret so the first real run is not during an incident.

Maturity levels for pipeline secrets

LevelWhat it looks like
1. Ad hocSecrets in code, shared spreadsheets or long-lived CI variables used by every job
2. StoredSecrets in CI-native stores or a secret manager, masked in logs, but long-lived and broadly scoped
3. ScopedEnvironment-scoped secrets, separate build and deploy jobs, scanning at push and in CI
4. FederatedOIDC for clouds, dynamic credentials for databases, automated rotation for what remains
5. AuditedAccess logs monitored with alerts, periodic access reviews, tested leak runbook

Learn secure pipelines hands-on

OIDC federation, Vault integration, secret scanning and pipeline hardening are part of our DevSecOps training in Pune, delivered in the Kharadi classroom and live online across India. For the wider list of security tools used in pipelines, see 12 DevSecOps tools for secure CI/CD.

Frequently asked questions

What is the safest way to give a CI pipeline cloud access?

Use OIDC federation. The CI system issues a signed token per job, and the cloud exchanges it for temporary credentials under a role whose trust policy only accepts that repository and environment. No access key is stored.

Are CI/CD secret variables safe enough?

They are encrypted and masked in logs, which is fine for a few values. Scope them to protected environments, keep deploy credentials away from build jobs, and use a central secret manager when many pipelines need the same secrets.

How often should secrets be rotated?

Prefer credentials that expire on their own. For static secrets, automate rotation on a schedule your security policy sets, and always rotate immediately when a leak is suspected or someone with access leaves.

Why are secrets in Docker images dangerous?

Every layer is kept in the image, so a secret copied in and later deleted is still readable from the image history. Use BuildKit secret mounts so the value is available only during the build step.

Does masking in CI logs prevent leaks?

Only partly. Masking replaces exact matches of registered values. Encoded, split or transformed values are not masked, and a malicious step can send the secret elsewhere.

What should I do if a secret is pushed to GitHub?

Revoke it at the issuer first, then rotate and update consumers. Check audit logs for use of the old value. Rewriting Git history is optional cleanup and does not undo exposure.

Which secret manager should a DevOps team use?

Use your cloud provider's manager if you are on one cloud. HashiCorp Vault suits multi-cloud or on-premises setups and adds dynamic secrets. The bigger gain comes from having one manager rather than which one you pick.

References

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Angry Angry 0
Sad Sad 0
Wow Wow 0
Mridul I am a passionate technology enthusiast with a strong focus on DevOps, Cloud Computing, and Cybersecurity. Through my blogs at DevOps Training Institute, I aim to simplify complex concepts and share practical insights for learners and professionals. My goal is to empower readers with knowledge, hands-on tips, and industry best practices to stay ahead in the ever-evolving world of DevOps.