Secrets and Sensitive Data in Kubernetes: Secure Management with Secrets, Vault, and Sealed-Secrets

Introduction: Why Native Kubernetes Secrets Are Insecure by Default

Kubernetes Secrets are the first thing a DevOps engineer reaches for when passing a database password or API key into a Pod. However, most teams underestimate the risks: by default, secrets are stored in etcd in Base64 format — this is not encryption, it's merely encoding. Anyone who gains access to an etcd backup or to a Secret object via kubectl can immediately read the data in plain text.

Real-world threats include:

• etcd compromise — an attacker with access to an etcd snapshot obtains all cluster secrets.
• Excessive RBAC permissions — developers accidentally gain get secrets rights in production namespaces.
• Secrets in Git — a manifest with encoded data is committed to a repository.
• No rotation — the same password lives for years.

In 2026, secrets management in Kubernetes is not optional — it's a mandatory component of a secure infrastructure. Let's explore the approaches and best practices.

Overview of Secrets Management Approaches

1. Native Kubernetes Secrets

A built-in mechanism that is easy to use but requires additional protection:

• Enable encryption at rest via EncryptionConfiguration on the API server.
• Restrict RBAC: minimal permission sets at the namespace level.
• Enable Audit Logging to track access.

2. HashiCorp Vault

A fully-featured secrets store with dynamic credentials, a lease mechanism, and detailed auditing. It integrates with Kubernetes via Agent Injector (sidecar) or CSI Provider. Best suited for large teams with high security requirements.

3. Sealed Secrets (Bitnami)

A Kubernetes controller that allows you to safely store encrypted secrets directly in Git. The secret is encrypted with the cluster's public key and can only be decrypted by the cluster itself. Ideal for a GitOps workflow.

4. External Secrets Operator

An operator that synchronizes secrets from external stores (AWS Secrets Manager, GCP Secret Manager, HashiCorp Vault, Azure Key Vault) into native Kubernetes Secrets. Enables centralized secrets management outside the cluster.

Hands-On: Setting Up Sealed Secrets in a Cluster

Sealed Secrets is the optimal choice for teams practicing GitOps. Let's walk through step-by-step installation and usage.

Step 1: Install the Controller

helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets\nhelm repo update\nhelm install sealed-secrets sealed-secrets/sealed-secrets \\\n  --namespace kube-system \\\n  --set fullnameOverride=sealed-secrets-controller

Step 2: Install the kubeseal CLI

# Linux\nwget https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.26.0/kubeseal-0.26.0-linux-amd64.tar.gz\ntar xvf kubeseal-0.26.0-linux-amd64.tar.gz\nsudo mv kubeseal /usr/local/bin/

Step 3: Create and Encrypt a Secret

First, create a regular Secret manifest, then encrypt it with kubeseal:

kubectl create secret generic db-credentials \\\n  --from-literal=DB_PASSWORD=supersecret123 \\\n  --from-literal=DB_USER=appuser \\\n  --namespace=production \\\n  --dry-run=client -o yaml | \\\n  kubeseal --controller-name=sealed-secrets-controller \\\n           --controller-namespace=kube-system \\\n           --format yaml > sealed-db-credentials.yaml

The resulting manifest sealed-db-credentials.yaml looks like this:

apiVersion: bitnami.com/v1alpha1\nkind: SealedSecret\nmetadata:\n  name: db-credentials\n  namespace: production\nspec:\n  encryptedData:\n    DB_PASSWORD: AgBy3i4OJSWK+PiTySYZZA9rO43cGDEq...\n    DB_USER: AgAKIBDLs8yHFUVCG+p1k2M3N4...\n  template:\n    metadata:\n      name: db-credentials\n      namespace: production\n    type: Opaque

This file is safe to commit to Git — without access to the cluster's private key, decrypting it is impossible. Apply the manifest:

kubectl apply -f sealed-db-credentials.yaml

The controller will automatically create a native Secret in the production namespace.

Integrating HashiCorp Vault with Kubernetes

Agent Injector

Vault Agent Injector works as a mutating admission webhook: when a Pod is created with the required annotations, a sidecar container is automatically injected into it, which authenticates to Vault and mounts secrets into the Pod's filesystem.

Example Pod annotations:

apiVersion: v1\nkind: Pod\nmetadata:\n  name: go-app\n  namespace: production\n  annotations:\n    vault.hashicorp.com/agent-inject: "true"\n    vault.hashicorp.com/role: "go-app-role"\n    vault.hashicorp.com/agent-inject-secret-config.env: "secret/data/production/go-app"\n    vault.hashicorp.com/agent-inject-template-config.env: |\n      {{- with secret "secret/data/production/go-app" -}}\n      export DB_PASSWORD={{ .Data.data.db_password }}\n      export API_KEY={{ .Data.data.api_key }}\n      {{- end }}\nspec:\n  serviceAccountName: go-app-sa\n  containers:\n  - name: go-app\n    image: myregistry/go-app:latest

CSI Provider

Vault CSI Provider mounts secrets as regular files via the Kubernetes Secrets Store CSI Driver — without a sidecar container, reducing overhead. It is well-suited for cases where the application already reads configuration from files.

apiVersion: secrets-store.csi.x-k8s.io/v1\nkind: SecretProviderClass\nmetadata:\n  name: vault-db-credentials\n  namespace: production\nspec:\n  provider: vault\n  parameters:\n    vaultAddress: "https://vault.example.com"\n    roleName: "go-app-role"\n    objects: |\n      - objectName: "db_password"\n        secretPath: "secret/data/production/go-app"\n        secretKey: "db_password"

Configuring Kubernetes Auth in Vault

# Enable Kubernetes auth\nvault auth enable kubernetes\n\n# Configure cluster connection\nvault write auth/kubernetes/config \\\n  kubernetes_host="https://kubernetes.default.svc" \\\n  kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \\\n  token_reviewer_jwt=@/var/run/secrets/kubernetes.io/serviceaccount/token\n\n# Create a role for the application\nvault write auth/kubernetes/role/go-app-role \\\n  bound_service_account_names=go-app-sa \\\n  bound_service_account_namespaces=production \\\n  policies=go-app-policy \\\n  ttl=1h

Secrets Management in the CI/CD Pipeline

GitHub Actions

In GitHub Actions, secrets are stored in repository or organization settings and passed into workflows via environment variables. Never print secrets to logs.

name: Deploy to Kubernetes\non:\n  push:\n    branches: [main]\njobs:\n  deploy:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - name: Configure kubectl\n        uses: azure/k8s-set-context@v3\n        with:\n          kubeconfig: ${{ secrets.KUBECONFIG }}\n      - name: Deploy application\n        env:\n          REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}\n        run: |\n          echo "$REGISTRY_TOKEN" | docker login registry.example.com -u ci --password-stdin\n          kubectl apply -f k8s/

GitLab CI

In GitLab CI, secrets are defined as Protected and Masked variables in the project settings. For HashiCorp Vault integration, GitLab supports a native JWT mechanism:

deploy:production:\n  stage: deploy\n  id_tokens:\n    VAULT_ID_TOKEN:\n      aud: https://vault.example.com\n  secrets:\n    DB_PASSWORD:\n      vault: production/go-app/db_password@secret\n      file: false\n  script:\n    - kubectl create secret generic db-credentials \\\n        --from-literal=DB_PASSWORD="$DB_PASSWORD" \\\n        --namespace=production \\\n        --dry-run=client -o yaml | kubectl apply -f -

This approach allows secrets to be retrieved directly from Vault without storing them in GitLab, which aligns with the principle of least trust.

Secret Rotation Without Downtime

Secret rotation is a critically important process that is often overlooked until the first incident. Let's look at a strategy for Go and PHP applications.

Double-Write Strategy

1. A new version of the secret is created in Vault, while the old one remains active.
2. The application is restarted with the new secret (rolling update in Kubernetes).
3. After a successful deployment, the old version is marked as deprecated.

Go Application: Hot Configuration Reload

Go applications can use the Vault SDK for dynamic credential updates without a restart:

// Simplified lease monitoring example\nfunc renewSecret(client *vault.Client, secret *vault.Secret) {\n    renewer, _ := client.NewLifetimeWatcher(&vault.LifetimeWatcherInput{\n        Secret: secret,\n        Increment: 3600,\n    })\n    go renewer.Start()\n    defer renewer.Stop()\n    for {\n        select {\n        case renewal := <-renewer.RenewCh():\n            log.Printf("Secret renewed: %v", renewal.Secret.LeaseDuration)\n        case err := <-renewer.DoneCh():\n            log.Printf("Renewal failed, fetching new secret: %v", err)\n            // Fetch new secret\n            return\n        }\n    }\n}

PHP Application: Rotation via Kubernetes Rolling Update

PHP applications typically do not maintain long-lived connections, so a standard Kubernetes rolling update handles the task without downtime:

apiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: php-app\n  namespace: production\nspec:\n  replicas: 3\n  strategy:\n    type: RollingUpdate\n    rollingUpdate:\n      maxUnavailable: 0\n      maxSurge: 1\n  template:\n    spec:\n      containers:\n      - name: php-app\n        image: myregistry/php-app:latest\n        envFrom:\n        - secretRef:\n            name: db-credentials

When rotating a secret, simply update the Secret object and run kubectl rollout restart deployment/php-app -n production.

Auditing and Monitoring Secret Access

Security without monitoring is an illusion. Configure the following mechanisms:

Kubernetes Audit Logging

Enable auditing at the API server level, capturing requests to the secrets resource:

apiVersion: audit.k8s.io/v1\nkind: Policy\nrules:\n- level: Metadata\n  resources:\n  - group: ""\n    resources: ["secrets"]\n  verbs: ["get", "list", "watch"]\n- level: RequestResponse\n  resources:\n  - group: ""\n    resources: ["secrets"]\n  verbs: ["create", "update", "patch", "delete"]

Vault Audit Devices

In HashiCorp Vault, enable a file or syslog audit device:

vault audit enable file file_path=/var/log/vault/audit.log

All secret accesses are recorded with the client, timestamp, and operation result. Integrate logs with ELK Stack or Grafana Loki for centralized analysis.

Alerting

Set up alerts for anomalous events: bulk listing of secrets, access from unexpected namespaces, access attempts with revoked tokens.

Security Checklist for a Production Cluster

• Encryption at rest — enable EncryptionConfiguration for etcd with an AES-CBC or KMS provider.
• RBAC minimalism — principle of least privilege: roles scoped to required namespaces and resources only, no wildcards.
• No plaintext secrets in Git — use Sealed Secrets or External Secrets Operator.
• Audit logs enabled — at minimum Metadata level for secrets resources.
• Rotation configured — automatic rotation via Vault leases or an external scheduler.
• Network Policy — restrict network access to Vault and other secrets stores.
• ServiceAccount tokens with limited TTL — use Bound Service Account Tokens (not static ones).
• Disable default ServiceAccount token mounting — set automountServiceAccountToken: false in Deployments where the token is not needed.
• Image scanning — scan Docker images for hardcoded secrets (truffleHog, gitleaks in CI/CD).
• Anomaly monitoring — alerts for unusual behavior when accessing secrets.

Conclusion

Secure secrets management in Kubernetes is a multi-layered challenge that requires choosing the right tools for the specific context. For small teams using a GitOps approach, Sealed Secrets addresses most risks. For enterprise infrastructure with compliance requirements, HashiCorp Vault provides full control: dynamic credentials, detailed auditing, and centralized policy management. External Secrets Operator flexibly bridges Kubernetes with cloud-based secrets stores.

Regardless of the tool chosen, the security foundation rests on three pillars: least privilege, auditing all access, and regular rotation. Implement these practices today — and your cluster will be ready for the security demands of 2026.