DevOps

Zero-Downtime Deployment in Kubernetes: Rolling Updates, Canary, and Blue-Green in Practice

Ruslan Ismailov Published 14 min read
Z

Introduction: What Is Zero-Downtime Deployment and Why It Matters

Every time you ship a new version of your application, you face a risk: users may encounter errors, service unavailability, or unpredictable behavior. Zero-downtime deployment is a set of practices and technical approaches that allow you to update a production environment without interrupting request handling.

In the context of Kubernetes, zero-downtime deployment is no longer an exotic technique — it's a baseline requirement for any serious product. The platform provides built-in mechanisms (rolling updates) and also supports more advanced strategies: blue-green and canary. In this article, we'll walk through each of them with real manifests and commands.

This article is aimed at DevOps engineers, SREs, and backend developers who already work with Kubernetes and want to build reliable delivery pipelines.

Rolling Update in Kubernetes: Configuration and Pitfalls

Rolling update is the default deployment strategy in Kubernetes. New pods are launched gradually while old ones are terminated as new ones become ready. This keeps the service available throughout the entire update process.

Key Parameters: maxSurge and maxUnavailable

Rolling update behavior is controlled by two parameters in the strategy section of the Deployment manifest:

  • maxSurge — the maximum number of pods that can exist above the desired replica count simultaneously. Can be set as a number or percentage.
  • maxUnavailable — the maximum number of pods that can be unavailable during the update process.

Example Deployment manifest with a configured rolling update:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: production
spec:
  replicas: 4
  selector:
    matchLabels:
      app: my-app
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: my-app
        version: v2
    spec:
      containers:
        - name: my-app
          image: my-registry/my-app:v2
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health/ready
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
            failureThreshold: 3
          livenessProbe:
            httpGet:
              path: /health/live
              port: 8080
            initialDelaySeconds: 15
            periodSeconds: 20

Readiness and Liveness Probes — A Mandatory Element

Without properly configured probes, rolling update loses its purpose. A readiness probe tells Kubernetes that a pod is ready to receive traffic. Until the probe passes, the pod is excluded from load balancing. A liveness probe determines whether the process is alive — if not, the pod is restarted.

A common mistake: the application starts but hasn't finished initialization yet (cache warm-up, migrations). Without a readiness probe, Kubernetes will start routing traffic to an unready instance. Also use startupProbe for applications with long startup times.

kubectl Commands for Managing Rolling Updates

# Update the image
kubectl set image deployment/my-app my-app=my-registry/my-app:v2 -n production

# Watch the rollout progress
kubectl rollout status deployment/my-app -n production

# View rollout history
kubectl rollout history deployment/my-app -n production

# Roll back to the previous version
kubectl rollout undo deployment/my-app -n production

Blue-Green Deployment: Switching Traffic via Service and Ingress

Blue-green deployment involves maintaining two identical environments: blue (the current production version) and green (the new version). Traffic is directed to only one of them at any given time. Once the green environment is successfully tested, you switch traffic with a single command.

Blue and Green Deployment Manifests

# Blue Deployment (current production)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-blue
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
      slot: blue
  template:
    metadata:
      labels:
        app: my-app
        slot: blue
        version: v1
    spec:
      containers:
        - name: my-app
          image: my-registry/my-app:v1
          ports:
            - containerPort: 8080
---
# Green Deployment (new version)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-green
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
      slot: green
  template:
    metadata:
      labels:
        app: my-app
        slot: green
        version: v2
    spec:
      containers:
        - name: my-app
          image: my-registry/my-app:v2
          ports:
            - containerPort: 8080

Service and Traffic Switching

apiVersion: v1
kind: Service
metadata:
  name: my-app-svc
  namespace: production
spec:
  selector:
    app: my-app
    slot: blue   # Change to "green" to switch traffic
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

Traffic switching is done by patching the service:

kubectl patch service my-app-svc -n production \
  -p '{"spec":{"selector":{"slot":"green"}}}'

When using Ingress with NGINX Ingress Controller or similar tools, you can switch traffic at the Ingress resource level by changing the backend service. This provides additional control without modifying the Service itself.

Advantage of blue-green: instant switching and equally instant rollback. Disadvantage: doubled resource usage during the deployment period.

Canary Deployment: Gradual Rollout with Traffic Weight Rules

Canary deployment lets you route a small percentage of traffic to the new version of your application without exposing all users to the risk. It's the ideal strategy for testing changes under real production load.

Canary with Native Kubernetes (No Service Mesh)

The simplest approach is to run a canary deployment with a small number of replicas alongside a stable deployment. Kubernetes distributes traffic across pods proportionally to their count.

# Stable Deployment: 9 replicas = ~90% of traffic
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-stable
  namespace: production
spec:
  replicas: 9
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
        track: stable
    spec:
      containers:
        - name: my-app
          image: my-registry/my-app:v1
---
# Canary Deployment: 1 replica = ~10% of traffic
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-canary
  namespace: production
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
        track: canary
    spec:
      containers:
        - name: my-app
          image: my-registry/my-app:v2

The Service selects pods only by the app: my-app label, covering both deployments.

Canary via Ingress with Weight Annotations (NGINX)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-canary-ingress
  namespace: production
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
spec:
  rules:
    - host: my-app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app-canary-svc
                port:
                  number: 80

The canary-weight: "10" annotation routes 10% of traffic to the canary. The value is gradually increased — 10%, 25%, 50%, 100% — as confidence in the stability of the new version grows.

Canary with Argo Rollouts

For advanced canary deployment management, it's recommended to use Argo Rollouts — a Kubernetes extension that adds a Rollout resource type with built-in support for canary, blue-green, and metrics-based analysis.

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: my-app
  namespace: production
spec:
  replicas: 10
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause: {duration: 5m}
        - setWeight: 30
        - pause: {duration: 10m}
        - setWeight: 60
        - pause: {duration: 10m}
        - setWeight: 100
      analysis:
        templates:
          - templateName: success-rate
        startingStep: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: my-app
          image: my-registry/my-app:v2
          ports:
            - containerPort: 8080

Integrating Deployment Strategies into a CI/CD Pipeline

Zero-downtime deployment strategies only reach their full potential when combined with an automated CI/CD pipeline. Here's a typical flow using GitLab CI as an example:

stages:
  - build
  - test
  - deploy-canary
  - promote
  - rollback

build:
  stage: build
  script:
    - docker build -t my-registry/my-app:$CI_COMMIT_SHA .
    - docker push my-registry/my-app:$CI_COMMIT_SHA

deploy-canary:
  stage: deploy-canary
  script:
    - kubectl set image deployment/my-app-canary
        my-app=my-registry/my-app:$CI_COMMIT_SHA -n production
    - kubectl rollout status deployment/my-app-canary -n production
  environment:
    name: production/canary

promote:
  stage: promote
  when: manual
  script:
    - kubectl set image deployment/my-app-stable
        my-app=my-registry/my-app:$CI_COMMIT_SHA -n production
    - kubectl rollout status deployment/my-app-stable -n production

rollback:
  stage: rollback
  when: manual
  script:
    - kubectl rollout undo deployment/my-app-stable -n production
    - kubectl rollout undo deployment/my-app-canary -n production

Key integration principles:

  • The Docker image tag is tied to the commit ($CI_COMMIT_SHA), ensuring reproducibility.
  • Each deployment step is verified via kubectl rollout status.
  • Promote (full rollout) is performed manually after observing canary metrics.
  • Rollback is available as a separate pipeline stage.

Rollback: Manual and Automatic

Even a perfect deployment can go wrong. Kubernetes stores the revision history of a Deployment, allowing you to roll back quickly.

Manual Rollback

# Roll back to the previous revision
kubectl rollout undo deployment/my-app -n production

# Roll back to a specific revision
kubectl rollout undo deployment/my-app --to-revision=3 -n production

# View revisions with details
kubectl rollout history deployment/my-app -n production --revision=2

Automatic Rollback via Argo Rollouts and Metrics Analysis

Argo Rollouts supports AnalysisTemplate — a definition of deployment success conditions based on metrics from Prometheus:

apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
  namespace: production
spec:
  metrics:
    - name: success-rate
      interval: 1m
      successCondition: result[0] >= 0.95
      failureLimit: 3
      provider:
        prometheus:
          address: http://prometheus:9090
          query: |
            sum(rate(http_requests_total{status!~"5..",app="my-app"}[1m]))
            /
            sum(rate(http_requests_total{app="my-app"}[1m]))

If the success rate drops below 95% three times in a row, Argo Rollouts automatically aborts the deployment and rolls back to the previous version.

It's also useful to configure progressDeadlineSeconds in the Deployment — if the deployment doesn't complete within the specified time, Kubernetes marks it as failed:

spec:
  progressDeadlineSeconds: 300

Deployment Monitoring: How to Know the Update Succeeded

Zero-downtime deployment is impossible without observability. Here's what to monitor:

  • Error rate — the percentage of 5xx responses. A sudden spike signals a problem.
  • Latency (p50, p95, p99) — degraded response times may indicate a regression.
  • Pod restart count — frequent restarts suggest application crashes.
  • Readiness probe failures — pods failing the readiness check.
  • Kubernetes events — kubectl get events -n production --sort-by=.lastTimestamp.

Useful Commands for Real-Time Deployment Monitoring

# Deployment status
kubectl rollout status deployment/my-app -n production

# Pod status
kubectl get pods -n production -l app=my-app -w

# Pod description when troubleshooting
kubectl describe pod <pod-name> -n production

# Live log streaming
kubectl logs -f -l app=my-app -n production --tail=100

# Namespace events
kubectl get events -n production --sort-by=.lastTimestamp

Integrate Prometheus + Grafana with deployment dashboards. A good practice is to automatically annotate the Grafana dashboard on each deployment — this allows you to visually correlate metric changes with the moment of the update.

Comparing Deployment Strategies

Each strategy suits different scenarios. Here are the key characteristics:

  • Rolling Update: built into Kubernetes, minimal resource overhead, suitable for most services. The downside is that two versions run simultaneously during the update, which requires backward compatibility of APIs and database schemas.
  • Blue-Green: instant switching and rollback, ideal for critical systems. Requires doubled resources during deployment. More complex with stateful components.
  • Canary: the safest strategy — real traffic tests the new version on a small subset of users. Requires solid monitoring and traffic weight management tools (Argo Rollouts, Istio, NGINX). Best suited for high-load systems with heavy traffic.

Recommendations for choosing:

  1. Start with Rolling Update — it's a sensible default for most applications.
  2. Move to Blue-Green when you need a guaranteed instant rollback and have the resources to spare.
  3. Adopt Canary for high-load services where even 1% of errors is critical and you have a mature monitoring system in place.

Conclusion

Zero-downtime deployment in Kubernetes is not just a technical configuration — it's an engineering culture. Rolling update is the right starting point for most teams. Blue-green provides confidence during critical updates. Canary is the most mature strategy for high-load production systems.

Any of these strategies works reliably only in combination with properly configured readiness/liveness probes, an automated CI/CD pipeline, and comprehensive monitoring. Invest time in setting up these components — they'll pay off with peaceful nights and confident releases.

The best deployment is the one users never notice.

Technologies

Tags

Ruslan Ismailov

Senior Web / Backend Developer. Senior web/backend developer with 9 years of experience. Stack: PHP, Laravel, PostgreSQL, Redis, Docker, Kubernetes, REST, microservices, CI/CD. More about me →