DevOps

Building a Multilingual CI/CD Pipeline for a Monorepo with Go and PHP Services in Kubernetes

Ruslan Ismailov Published 12 min read
B

Introduction: Monorepo Challenges

A monorepo is a popular architectural strategy where multiple services written in different programming languages live within a single Git repository. Companies like Google, Meta, and Uber have used the monorepo approach for years. However, along with the convenience of dependency management and unified code review come serious CI/CD challenges: how do you avoid rebuilding all services when only one changes? How do you organize parallel pipelines for Go and PHP? How do you manage deployments to Kubernetes without duplicating configurations?

In this article, we'll walk through a practical architecture for a multilingual CI/CD pipeline in a monorepo where Go and PHP microservices coexist and deploy to a Kubernetes cluster. The target audience is DevOps engineers and tech leads who have already experienced the pain of "a green pipeline taking 40 minutes because of a single-line change."

Monorepo Structure: Directory Organization

A well-designed directory structure is the foundation of a manageable monorepo. Separate services by language and domain, and move shared components into dedicated directories.

monorepo/
├── services/
│   ├── go/
│   │   ├── auth-service/
│   │   │   ├── cmd/
│   │   │   ├── internal/
│   │   │   ├── Dockerfile
│   │   │   └── go.mod
│   │   └── payment-service/
│   │       ├── cmd/
│   │       ├── internal/
│   │       ├── Dockerfile
│   │       └── go.mod
│   └── php/
│       ├── api-gateway/
│       │   ├── src/
│       │   ├── Dockerfile
│       │   └── composer.json
│       └── billing-service/
│           ├── src/
│           ├── Dockerfile
│           └── composer.json
├── infra/
│   ├── helm/
│   │   ├── auth-service/
│   │   ├── payment-service/
│   │   ├── api-gateway/
│   │   └── billing-service/
│   └── kustomize/
│       ├── base/
│       └── overlays/
│           ├── staging/
│           └── production/
├── .github/
│   └── workflows/
├── scripts/
│   └── detect-changes.sh
└── Makefile

The key principle: each service is self-contained — it includes its own Dockerfile, dependency configuration, and tests. Shared libraries are placed in a libs/ directory with explicit versions, so that a change to a shared library only triggers a rebuild of the services that depend on it.

Detecting Changed Services: Path Filtering

Path filtering is the key technique for CI/CD in a monorepo. The goal is to trigger the pipeline only for services whose files have changed in a commit or pull request.

Path Filtering in GitHub Actions

GitHub Actions supports a native paths filter at the workflow level, but for dynamic change detection it's better to use dorny/paths-filter:

name: CI Pipeline
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  detect-changes:
    runs-on: ubuntu-latest
    outputs:
      auth-service: ${{ steps.filter.outputs.auth-service }}
      payment-service: ${{ steps.filter.outputs.payment-service }}
      api-gateway: ${{ steps.filter.outputs.api-gateway }}
      billing-service: ${{ steps.filter.outputs.billing-service }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            auth-service:
              - 'services/go/auth-service/**'
              - 'libs/go-common/**'
            payment-service:
              - 'services/go/payment-service/**'
              - 'libs/go-common/**'
            api-gateway:
              - 'services/php/api-gateway/**'
              - 'libs/php-shared/**'
            billing-service:
              - 'services/php/billing-service/**'
              - 'libs/php-shared/**'

  build-auth-service:
    needs: detect-changes
    if: needs.detect-changes.outputs.auth-service == 'true'
    uses: ./.github/workflows/build-go-service.yml
    with:
      service: auth-service
      path: services/go/auth-service

Path Filtering in GitLab CI

In GitLab CI, the changes directive is used within the rules section:

build-auth-service:
  stage: build
  rules:
    - changes:
        - services/go/auth-service/**/*
        - libs/go-common/**/*
  script:
    - docker build -t $CI_REGISTRY_IMAGE/auth-service:$CI_COMMIT_SHORT_SHA
        services/go/auth-service/

build-api-gateway:
  stage: build
  rules:
    - changes:
        - services/php/api-gateway/**/*
        - libs/php-shared/**/*
  script:
    - docker build -t $CI_REGISTRY_IMAGE/api-gateway:$CI_COMMIT_SHORT_SHA
        services/php/api-gateway/

Independent Docker Builds: Go and PHP

Multistage Dockerfile for Go Services

For Go services, the optimal strategy is a multistage build with a final distroless image. This minimizes the image size and reduces the attack surface:

# services/go/auth-service/Dockerfile
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build \
    -ldflags="-w -s -X main.version=${VERSION}" \
    -o auth-service ./cmd/server

FROM gcr.io/distroless/static-debian12 AS production
COPY --from=builder /app/auth-service /auth-service
USER nonroot:nonroot
ENTRYPOINT ["/auth-service"]

Distroless images contain no shell, package manager, or other tools, making them significantly more secure for production environments.

Dockerfile for PHP Services (FPM + Nginx)

PHP services require two containers: PHP-FPM for request processing and Nginx as a reverse proxy. We use a multistage build with Composer optimization:

# services/php/api-gateway/Dockerfile
FROM composer:2.7 AS composer-deps
WORKDIR /app
COPY composer.json composer.lock ./
RUN composer install \
    --no-dev \
    --no-scripts \
    --prefer-dist \
    --optimize-autoloader

FROM php:8.3-fpm-alpine AS production
RUN apk add --no-cache \
    redis \
    libpq-dev \
    && docker-php-ext-install pdo pgsql opcache

COPY --from=composer-deps /app/vendor /var/www/html/vendor
COPY src/ /var/www/html/src/
COPY config/ /var/www/html/config/
COPY php.ini /usr/local/etc/php/conf.d/custom.ini

FROM nginx:1.25-alpine AS nginx
COPY nginx/default.conf /etc/nginx/conf.d/default.conf
COPY --from=production /var/www/html/public /var/www/html/public

Image Version Management: Tagging and Semantic Versioning

A solid Docker image tagging strategy is critical for deployment traceability. We recommend using multiple tags simultaneously:

  • Git SHA — a unique identifier for each commit: auth-service:a1b2c3d
  • Branch + SHA — for isolating feature branches: auth-service:feature-oauth-a1b2c3d
  • Semantic version — for releases: auth-service:v1.4.2
  • latest — only for the main/master branch in staging
#!/bin/bash
# scripts/tag-image.sh
SERVICE=$1
GIT_SHA=$(git rev-parse --short HEAD)
BRANCH=$(git rev-parse --abbrev-ref HEAD)
REGISTRY="registry.company.com"

BASE_TAG="$REGISTRY/$SERVICE"

# Always tag by SHA
docker tag $BASE_TAG:build $BASE_TAG:$GIT_SHA

# For main — additionally tag with latest and semantic version
if [ "$BRANCH" = "main" ]; then
  VERSION=$(cat services/$SERVICE/VERSION)
  docker tag $BASE_TAG:build $BASE_TAG:$VERSION
  docker tag $BASE_TAG:build $BASE_TAG:latest
fi

docker push $BASE_TAG --all-tags

Parallel Pipelines: Matrix Jobs and Dependencies

GitHub Actions supports matrix strategies that allow multiple services of the same type to be built in parallel. This significantly reduces the overall CI/CD pipeline duration.

build-go-services:
  needs: detect-changes
  strategy:
    matrix:
      service:
        - name: auth-service
          changed: ${{ needs.detect-changes.outputs.auth-service }}
        - name: payment-service
          changed: ${{ needs.detect-changes.outputs.payment-service }}
    fail-fast: false
  runs-on: ubuntu-latest
  if: matrix.service.changed == 'true'
  steps:
    - uses: actions/checkout@v4
    - name: Build Go service
      run: |
        docker build \
          --build-arg VERSION=${{ github.sha }} \
          -t $REGISTRY/${{ matrix.service.name }}:${{ github.sha }} \
          services/go/${{ matrix.service.name }}/
    - name: Push image
      run: docker push $REGISTRY/${{ matrix.service.name }}:${{ github.sha }}

deploy:
  needs: [build-go-services, build-php-services, security-scan]
  if: github.ref == 'refs/heads/main'
  runs-on: ubuntu-latest
  steps:
    - name: Deploy to Kubernetes
      run: ./scripts/deploy.sh

Use fail-fast: false in the matrix so that a problem with one service doesn't block the build of the others. This is especially important when you have a large number of independent microservices.

Deploying to Kubernetes: Helm, Kustomize, and Namespace Strategies

Helm Charts for Multi-Service Deployment

A separate Helm chart is created for each service, sharing a common values.yaml template. Overridable values — the image and tag — are passed through CI/CD:

# infra/helm/auth-service/values.yaml
image:
  repository: registry.company.com/auth-service
  tag: latest
  pullPolicy: IfNotPresent

replicaCount: 2

resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 500m
    memory: 256Mi

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70
# scripts/deploy.sh
#!/bin/bash
SERVICE=$1
ENVIRONMENT=$2
IMAGE_TAG=$3

helm upgrade --install $SERVICE \
  infra/helm/$SERVICE \
  --namespace $ENVIRONMENT \
  --create-namespace \
  --set image.tag=$IMAGE_TAG \
  --set environment=$ENVIRONMENT \
  --values infra/helm/$SERVICE/values-$ENVIRONMENT.yaml \
  --wait \
  --timeout 5m0s

Namespace Strategy

The recommended Kubernetes namespace strategy for a monorepo:

  • staging — all services in the staging environment
  • production — the production environment with NetworkPolicy enforcement
  • feature-{branch-name} — temporary namespaces for feature branches (deleted after merge)

Kustomize is convenient for managing differences between environments without duplicating YAML configurations. Base manifests live in infra/kustomize/base/, while overlay directories contain only the overrides for a specific environment.

Shared Pipeline Steps: Linting, Testing, and Image Scanning

Linting and Tests

For Go, we use golangci-lint configured via .golangci.yml; for PHP — phpstan and phpcs. Tests run in parallel with image builds, saving valuable pipeline time.

Image Scanning with Trivy

Integrating Trivy into the CI/CD pipeline is a mandatory step for a production-ready monorepo. Scanning runs immediately after the image is built, before deployment:

security-scan:
  needs: [build-go-services, build-php-services]
  runs-on: ubuntu-latest
  strategy:
    matrix:
      service: [auth-service, payment-service, api-gateway, billing-service]
  steps:
    - name: Run Trivy vulnerability scanner
      uses: aquasecurity/trivy-action@master
      with:
        image-ref: registry.company.com/${{ matrix.service }}:${{ github.sha }}
        format: sarif
        output: trivy-results-${{ matrix.service }}.sarif
        severity: CRITICAL,HIGH
        exit-code: 1

    - name: Upload Trivy results to GitHub Security
      uses: github/codeql-action/upload-sarif@v3
      with:
        sarif_file: trivy-results-${{ matrix.service }}.sarif

Secrets Management in a Multi-Service Pipeline

Secrets management is one of the most sensitive challenges in a monorepo CI/CD setup. Different services require different secrets, and it's critical to prevent secrets from one service from leaking into another.

The recommended approach is the External Secrets Operator integrated with HashiCorp Vault or AWS Secrets Manager. Each service gains access only to its own secrets namespace via a Kubernetes ServiceAccount and RBAC:

# infra/helm/auth-service/templates/external-secret.yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: auth-service-secrets
spec:
  refreshInterval: 15m
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: auth-service-secrets
  data:
    - secretKey: JWT_SECRET
      remoteRef:
        key: services/auth-service
        property: jwt_secret
    - secretKey: DB_PASSWORD
      remoteRef:
        key: services/auth-service
        property: db_password

For CI/CD, use GitHub Actions secrets scoped to environments, separating staging and production. Never store secrets in the repository — not even encrypted ones. Store only references to external secret stores.

Pipeline Monitoring and Deployment Time Metrics

Without measurement, there's no optimization. Key metrics for a monorepo CI/CD pipeline:

  • Pipeline duration — total time from commit to deployment (target: under 10 minutes)
  • Build cache hit rate — percentage of Docker layer cache utilization (target: above 70%)
  • Deployment frequency — how many deployments per day/week
  • Change failure rate — percentage of deployments that caused incidents
  • Mean time to recovery (MTTR) — average time to recover from a failure

For metrics visualization, GitHub Actions integrates with Datadog or Grafana via plugins. GitLab CI has built-in pipeline analytics and job execution time reports. For Kubernetes deployments, configure tracking through Argo Rollouts with automatic rollback on degraded golden signals.

Example of collecting metrics via GitHub Actions step summary:

- name: Report deployment metrics
  run: |
    DEPLOY_DURATION=$(($(date +%s) - $START_TIME))
    echo "## Deployment Summary" >> $GITHUB_STEP_SUMMARY
    echo "| Service | Image Tag | Duration |" >> $GITHUB_STEP_SUMMARY
    echo "|---------|-----------|----------|" >> $GITHUB_STEP_SUMMARY
    echo "| $SERVICE | $IMAGE_TAG | ${DEPLOY_DURATION}s |" >> $GITHUB_STEP_SUMMARY
    
    # Send metric to Datadog
    curl -X POST "https://api.datadoghq.com/api/v1/series" \
      -H "DD-API-KEY: ${{ secrets.DATADOG_API_KEY }}" \
      -d "{\"series\": [{\"metric\": \"cicd.deploy.duration\",
           \"points\": [[$START_TIME, $DEPLOY_DURATION]],
           \"tags\": [\"service:$SERVICE\", \"env:production\"]}]}"

Conclusion and Recommendations

Building a multilingual CI/CD pipeline for a monorepo is a non-trivial task that requires a systematic approach. Here are the key takeaways:

  1. Invest in path filtering from day one. This is the foundational mechanism without which a monorepo CI/CD becomes a bottleneck.
  2. Standardize Dockerfile templates for Go (distroless) and PHP (FPM+Nginx) — this simplifies maintenance and onboarding of new developers.
  3. Use matrix jobs for parallel building and testing of similar services — this can reduce pipeline time by 3–5x.
  4. Separate secrets by service using the External Secrets Operator and the principle of least privilege.
  5. Track DORA metrics — deployment frequency, lead time, MTTR, and change failure rate — they reveal the true effectiveness of your pipeline.
  6. Automate rollback via Helm rollback or Argo Rollouts with a canary strategy to minimize deployment risk.

A great CI/CD pipeline for a monorepo is more than just build automation. It's a fast-feedback system that empowers the team to deliver changes confidently and frequently, regardless of how many languages or services live in the repository.

The architecture described here scales from 5 to 50+ services without fundamental changes. Start small — set up path filtering and parallel builds, then layer in security scanning and metrics. An iterative approach works just as well here as it does in product development.

Technologies

Tags

Ruslan Ismailov

Senior Web / Backend Developer. Senior web/backend developer with 9 years of experience. Stack: PHP, Laravel, PostgreSQL, Redis, Docker, Kubernetes, REST, microservices, CI/CD. More about me →