Building an Internal API Gateway with Docker and Go from Scratch: Routing, Load Balancing, and Circuit Breaker

Why Build Your Own API Gateway and When It Makes Sense

Off-the-shelf solutions — Kong, Traefik, AWS API Gateway — cover most use cases. But there are situations where building your own API Gateway in Go is a justified choice:

• You need full control over routing and load balancing logic without the overhead of someone else's code.
• Performance requirements are so specific that general-purpose solutions fall short.
• The team wants a deep understanding of how the gateway works internally, rather than managing a black box.
• The infrastructure is isolated and third-party dependencies are not allowed.
• You need custom circuit breaker logic or routing based on business rules.

If none of these conditions apply — use a ready-made solution. Otherwise, keep reading.

Solution Architecture: Components and Interaction Diagram

Our gateway consists of the following components:

• Config Loader — loads and hot-reloads route configuration from a YAML file without restarting the process.
• Router — matches incoming HTTP requests to routes by prefix/path.
• Load Balancer — selects a backend from the pool (round-robin or weighted round-robin).
• Circuit Breaker — tracks backend errors and temporarily removes a backend from rotation.
• Proxy — forwards the request to the selected backend and returns the response to the client.
• Metrics Collector — gathers statistics: RPS, latency, errors.

Interaction diagram (text description):

Client → [API Gateway: Router → Load Balancer → Circuit Breaker → Proxy] → Backend Service A / B / C → Response → Client

All components run inside a single Go process deployed in a Docker container. Backends are separate microservices in their own containers, connected via Docker Compose on a shared network.

Implementing Request Routing in Go with Dynamic Configuration

Let's start by defining the configuration in YAML:

routes:
  - prefix: /api/users
    backends:
      - url: http://users-service-1:8080
        weight: 2
      - url: http://users-service-2:8080
        weight: 1
  - prefix: /api/orders
    backends:
      - url: http://orders-service:8081
        weight: 1

Data structures for configuration:

package config

import (
    "os"
    "sync"
    "time"

    "gopkg.in/yaml.v3"
)

type Backend struct {
    URL    string `yaml:"url"`
    Weight int    `yaml:"weight"`
}

type Route struct {
    Prefix   string    `yaml:"prefix"`
    Backends []Backend `yaml:"backends"`
}

type Config struct {
    Routes []Route `yaml:"routes"`
}

type DynamicConfig struct {
    mu      sync.RWMutex
    current *Config
    path    string
}

func NewDynamicConfig(path string) (*DynamicConfig, error) {
    dc := &DynamicConfig{path: path}
    if err := dc.reload(); err != nil {
        return nil, err
    }
    go dc.watch()
    return dc, nil
}

func (dc *DynamicConfig) reload() error {
    data, err := os.ReadFile(dc.path)
    if err != nil {
        return err
    }
    var cfg Config
    if err := yaml.Unmarshal(data, &cfg); err != nil {
        return err
    }
    dc.mu.Lock()
    dc.current = &cfg
    dc.mu.Unlock()
    return nil
}

func (dc *DynamicConfig) watch() {
    ticker := time.NewTicker(10 * time.Second)
    for range ticker.C {
        _ = dc.reload()
    }
}

func (dc *DynamicConfig) Get() *Config {
    dc.mu.RLock()
    defer dc.mu.RUnlock()
    return dc.current
}

The router finds a route using longest-prefix match:

package router

import (
    "net/http"
    "sort"
    "strings"

    "gateway/config"
)

type Router struct {
    cfg *config.DynamicConfig
}

func New(cfg *config.DynamicConfig) *Router {
    return &Router{cfg: cfg}
}

func (r *Router) Match(req *http.Request) (*config.Route, bool) {
    routes := r.cfg.Get().Routes
    // sort by descending prefix length for longest-match
    sort.Slice(routes, func(i, j int) bool {
        return len(routes[i].Prefix) > len(routes[j].Prefix)
    })
    for _, route := range routes {
        if strings.HasPrefix(req.URL.Path, route.Prefix) {
            rv := route
            return &rv, true
        }
    }
    return nil, false
}

Load Balancing: Round-Robin and Weighted Round-Robin from Scratch

Round-robin is the simplest algorithm: each request goes to the next backend in sequence.

package balancer

import (
    "sync/atomic"

    "gateway/config"
)

type RoundRobin struct {
    counter uint64
}

func (rr *RoundRobin) Pick(backends []config.Backend) config.Backend {
    n := uint64(len(backends))
    idx := atomic.AddUint64(&rr.counter, 1) % n
    return backends[idx]
}

Weighted round-robin — backends with a higher weight receive proportionally more requests. We implement this by expanding the backend list:

package balancer

import (
    "sync/atomic"

    "gateway/config"
)

type WeightedRoundRobin struct {
    counter  uint64
    expanded []config.Backend
}

func NewWeightedRoundRobin(backends []config.Backend) *WeightedRoundRobin {
    var expanded []config.Backend
    for _, b := range backends {
        w := b.Weight
        if w <= 0 {
            w = 1
        }
        for i := 0; i < w; i++ {
            expanded = append(expanded, b)
        }
    }
    return &WeightedRoundRobin{expanded: expanded}
}

func (w *WeightedRoundRobin) Pick() config.Backend {
    n := uint64(len(w.expanded))
    idx := atomic.AddUint64(&w.counter, 1) % n
    return w.expanded[idx]
}

When the configuration is reloaded, the balancer is recreated with a new backend pool — atomically via sync/atomic and sync.RWMutex.

Implementing the Circuit Breaker Pattern in Go

The Circuit Breaker tracks the number of consecutive errors and transitions a backend to the OPEN state, temporarily removing it from rotation. After a specified timeout, a recovery attempt is made (the HALF-OPEN state).

package breaker

import (
    "sync"
    "time"
)

type State int

const (
    StateClosed   State = iota // normal operation
    StateOpen                  // backend unavailable
    StateHalfOpen              // checking recovery
)

type CircuitBreaker struct {
    mu           sync.Mutex
    state        State
    failures     int
    maxFailures  int
    timeout      time.Duration
    lastFailure  time.Time
}

func New(maxFailures int, timeout time.Duration) *CircuitBreaker {
    return &CircuitBreaker{
        maxFailures: maxFailures,
        timeout:     timeout,
        state:       StateClosed,
    }
}

func (cb *CircuitBreaker) Allow() bool {
    cb.mu.Lock()
    defer cb.mu.Unlock()

    switch cb.state {
    case StateClosed:
        return true
    case StateOpen:
        if time.Since(cb.lastFailure) > cb.timeout {
            cb.state = StateHalfOpen
            return true
        }
        return false
    case StateHalfOpen:
        return true
    }
    return false
}

func (cb *CircuitBreaker) RecordSuccess() {
    cb.mu.Lock()
    defer cb.mu.Unlock()
    cb.failures = 0
    cb.state = StateClosed
}

func (cb *CircuitBreaker) RecordFailure() {
    cb.mu.Lock()
    defer cb.mu.Unlock()
    cb.failures++
    cb.lastFailure = time.Now()
    if cb.failures >= cb.maxFailures {
        cb.state = StateOpen
    }
}

Each backend gets its own CircuitBreaker instance. Before proxying, we call Allow() — if it returns false, we skip that backend and pick the next one from the pool.

Integration with REST API Backends and Request Proxying

The main handler brings all components together:

package proxy

import (
    "io"
    "net/http"
    "net/url"
    "time"

    "gateway/balancer"
    "gateway/breaker"
    "gateway/config"
    "gateway/router"
)

type Handler struct {
    router   *router.Router
    breakers map[string]*breaker.CircuitBreaker
    client   *http.Client
}

func NewHandler(r *router.Router) *Handler {
    return &Handler{
        router:   r,
        breakers: make(map[string]*breaker.CircuitBreaker),
        client: &http.Client{
            Timeout: 10 * time.Second,
        },
    }
}

func (h *Handler) getBreaker(backendURL string) *breaker.CircuitBreaker {
    if cb, ok := h.breakers[backendURL]; ok {
        return cb
    }
    cb := breaker.New(5, 30*time.Second)
    h.breakers[backendURL] = cb
    return cb
}

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    route, ok := h.router.Match(r)
    if !ok {
        http.Error(w, "no route found", http.StatusNotFound)
        return
    }

    wrr := balancer.NewWeightedRoundRobin(route.Backends)
    var lastErr error

    for attempt := 0; attempt < len(route.Backends); attempt++ {
        backend := wrr.Pick()
        cb := h.getBreaker(backend.URL)

        if !cb.Allow() {
            continue
        }

        targetURL, err := url.Parse(backend.URL)
        if err != nil {
            continue
        }
        targetURL.Path = r.URL.Path
        targetURL.RawQuery = r.URL.RawQuery

        req, err := http.NewRequestWithContext(r.Context(), r.Method, targetURL.String(), r.Body)
        if err != nil {
            lastErr = err
            cb.RecordFailure()
            continue
        }
        req.Header = r.Header.Clone()

        resp, err := h.client.Do(req)
        if err != nil {
            lastErr = err
            cb.RecordFailure()
            continue
        }
        defer resp.Body.Close()

        if resp.StatusCode >= 500 {
            cb.RecordFailure()
            continue
        }

        cb.RecordSuccess()
        for k, v := range resp.Header {
            w.Header()[k] = v
        }
        w.WriteHeader(resp.StatusCode)
        io.Copy(w, resp.Body)
        return
    }

    if lastErr != nil {
        http.Error(w, "all backends unavailable: "+lastErr.Error(), http.StatusBadGateway)
    } else {
        http.Error(w, "all backends unavailable", http.StatusBadGateway)
    }
}

Containerizing the Gateway with Docker and Orchestrating via Docker Compose

The gateway Dockerfile uses a multi-stage build:

FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o gateway ./cmd/gateway

FROM alpine:3.19
RUN apk --no-cache add ca-certificates
WORKDIR /app
COPY --from=builder /app/gateway .
COPY config.yaml .
EXPOSE 8000
CMD ["./gateway"]

Docker Compose connects the gateway and backends on a shared network:

version: "3.9"

services:
  gateway:
    build: ./gateway
    ports:
      - "8000:8000"
    volumes:
      - ./config.yaml:/app/config.yaml
    networks:
      - internal
    depends_on:
      - users-service-1
      - users-service-2
      - orders-service

  users-service-1:
    image: mycompany/users-service:latest
    networks:
      - internal
    environment:
      - PORT=8080

  users-service-2:
    image: mycompany/users-service:latest
    networks:
      - internal
    environment:
      - PORT=8080

  orders-service:
    image: mycompany/orders-service:latest
    networks:
      - internal
    environment:
      - PORT=8081

networks:
  internal:
    driver: bridge

Thanks to the volume-mounted configuration, the dynamic reloader picks up changes to config.yaml without restarting the container.

Logging, Metrics, and Basic Monitoring

For logging, we use structured output via the standard log/slog package (Go 1.21+):

package middleware

import (
    "log/slog"
    "net/http"
    "time"
)

type responseWriter struct {
    http.ResponseWriter
    statusCode int
}

func (rw *responseWriter) WriteHeader(code int) {
    rw.statusCode = code
    rw.ResponseWriter.WriteHeader(code)
}

func Logging(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        rw := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}
        next.ServeHTTP(rw, r)
        slog.Info("request",
            "method", r.Method,
            "path", r.URL.Path,
            "status", rw.statusCode,
            "duration_ms", time.Since(start).Milliseconds(),
        )
    })
}

For metrics, we add a simple /metrics endpoint that exposes counters in a Prometheus-compatible format:

package metrics

import (
    "fmt"
    "net/http"
    "sync/atomic"
)

var (
    TotalRequests  uint64
    TotalErrors    uint64
    OpenBreakers   uint64
)

func Handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "# HELP gateway_requests_total Total requests\n")
    fmt.Fprintf(w, "gateway_requests_total %d\n", atomic.LoadUint64(&TotalRequests))
    fmt.Fprintf(w, "# HELP gateway_errors_total Total errors\n")
    fmt.Fprintf(w, "gateway_errors_total %d\n", atomic.LoadUint64(&TotalErrors))
    fmt.Fprintf(w, "# HELP gateway_open_breakers Open circuit breakers\n")
    fmt.Fprintf(w, "gateway_open_breakers %d\n", atomic.LoadUint64(&OpenBreakers))
}

The /metrics endpoint integrates easily with Prometheus + Grafana for visualization, providing the minimum viable monitoring for a production environment.

Limitations of This Approach and When to Use Ready-Made Solutions

A custom-built API Gateway is a deliberate trade-off. Here are the honest limitations:

• No built-in authentication: JWT, OAuth2, and API keys must be implemented manually or via middleware.
• No rate limiting out of the box: requires a separate module, often with Redis as the counter store.
• No UI or declarative DSL: YAML configuration is simpler than the Kong Admin API, but less convenient.
• Testing and maintenance: every feature requires writing tests. Bugs in the circuit breaker can lead to cascading failures.
• Scaling complexity: horizontal scaling of the gateway itself requires external storage for circuit breaker state.

When to use ready-made solutions:

• The team is small and lacks resources to maintain a custom gateway.
• You need a rich plugin ecosystem: request transformation, canary releases, mTLS.
• The infrastructure already runs on Kubernetes — Traefik or ingress-nginx will get the job done faster.
• The project is in its early stages and gateway requirements have not yet been finalized.

A custom API Gateway in Go is a valuable learning and production tool for teams that know exactly what they want and are ready to take full ownership of every line of code in a critical infrastructure component.