Laravel Queues in 2026: Horizontal Worker Scaling with Redis and Kubernetes

Introduction: When a Monolith Can No Longer Keep Up

Job queues are one of the first things to break under increased load. While traffic is moderate, a single php artisan queue:work process on the server handles the job. But when the number of queued jobs starts growing faster than workers can process them, classic problems emerge: delayed email delivery, stuck exports, piling up notifications, and overflowing queues.

In a monolithic architecture, horizontal worker scaling is painful. You have to manually spin up new processes, monitor their state, and balance the load. In 2026, the Laravel + Redis + Kubernetes stack solves this problem elegantly: you declare the desired state, and the infrastructure adjusts to the load automatically.

This article is a practical guide for mid/senior PHP developers who want to build a reliable, scalable background job processing system based on Laravel Queues, Redis Cluster, and Kubernetes with HPA.

Laravel Queues + Redis Architecture in 2026

Named Queues and Priorities

Laravel lets you organize multiple named queues with different priorities. Workers process queues in the order specified at startup:

php artisan queue:work redis --queue=critical,high,default,low --sleep=3 --tries=3 --max-time=3600

In the job class, priority is set via the $queue property or at dispatch time:

<?php

// Dispatching to a specific queue
SendInvoice::dispatch($order)->onQueue('critical');

// Or via the class property
class ProcessPayment implements ShouldQueue
{
    public string $queue = 'critical';
    public int $tries = 5;
    public int $timeout = 120;
    public int $backoff = 30;
}

Job Batching

Job Batching, introduced in Laravel 8, has become the standard in 2026 for parallel processing of large data volumes. A batch lets you dispatch thousands of jobs and receive a callback upon completion:

<?php

use Illuminate\\Bus\\Batch;
use Illuminate\\Support\\Facades\\Bus;

$batch = Bus::batch([
    new ImportChunk($chunk1),
    new ImportChunk($chunk2),
    new ImportChunk($chunk3),
])->then(function (Batch $batch) {
    // All jobs completed successfully
    ImportCompleted::dispatch($batch->id);
})->catch(function (Batch $batch, Throwable $e) {
    // First failure in the batch
    Log::error('Batch failed', ['batch_id' => $batch->id, 'error' => $e->getMessage()]);
})->finally(function (Batch $batch) {
    // Always runs
    Cache::forget('import_lock');
})->onQueue('high')
  ->dispatch();

queue.php Configuration for Redis

<?php
// config/queue.php

return [
    'default' => env('QUEUE_CONNECTION', 'redis'),

    'connections' => [
        'redis' => [
            'driver'      => 'redis',
            'connection'  => 'queue', // dedicated Redis connection
            'queue'       => env('REDIS_QUEUE', 'default'),
            'retry_after' => 90,
            'block_for'   => 5,   // blocking BLPOP instead of polling
            'after_commit' => true, // dispatch only after transaction commit
        ],
    ],
];

The after_commit parameter is critical in production: it ensures that a job is only added to the queue after the database transaction has been successfully committed.

Setting Up Redis Cluster as a Queue Backend

For high-load systems, a single Redis instance becomes a bottleneck. Redis Cluster distributes data across multiple nodes and provides fault tolerance.

Configure a dedicated Redis connection for queues in config/database.php:

<?php
// config/database.php

'redis' => [
    'client' => env('REDIS_CLIENT', 'phpredis'),

    'options' => [
        'cluster' => env('REDIS_CLUSTER', 'redis'),
        'prefix'  => env('REDIS_PREFIX', Str::slug(env('APP_NAME', 'laravel'), '_').'_database_'),
    ],

    'queue' => [
        'url'      => env('REDIS_URL'),
        'host'     => env('REDIS_HOST', '127.0.0.1'),
        'username' => env('REDIS_USERNAME'),
        'password' => env('REDIS_PASSWORD'),
        'port'     => env('REDIS_PORT', '6379'),
        'database' => env('REDIS_QUEUE_DB', '1'),
        // For Redis Cluster
        'cluster'  => [
            ['host' => 'redis-node-1', 'port' => 6379, 'password' => env('REDIS_PASSWORD')],
            ['host' => 'redis-node-2', 'port' => 6379, 'password' => env('REDIS_PASSWORD')],
            ['host' => 'redis-node-3', 'port' => 6379, 'password' => env('REDIS_PASSWORD')],
        ],
        'options'  => [
            'cluster' => 'redis',
        ],
    ],
],

Important note: when using Redis Cluster, all keys for a given queue must land in the same slot. Laravel automatically uses hash tags ({queue_name}) for this, but make sure your version of predis or the phpredis extension supports cluster mode.

Containerizing Workers: Dockerfile and Supervisor

Dockerfile for the Worker

FROM php:8.3-cli-alpine

# System dependencies
RUN apk add --no-cache \
    supervisor \
    git \
    curl \
    libpng-dev \
    oniguruma-dev \
    libxml2-dev

# PHP extensions
RUN docker-php-ext-install pdo_mysql mbstring exif pcntl bcmath

# phpredis for better performance
RUN pecl install redis && docker-php-ext-enable redis

# Opcache for CLI (speeds up worker startup)
RUN docker-php-ext-install opcache

# Composer
COPY --from=composer:2 /usr/bin/composer /usr/bin/composer

WORKDIR /var/www

COPY composer.json composer.lock ./
RUN composer install --no-dev --optimize-autoloader --no-interaction

COPY . .

# Supervisor config
COPY docker/supervisor/worker.conf /etc/supervisor/conf.d/worker.conf

# Permissions
RUN chown -R www-data:www-data /var/www/storage /var/www/bootstrap/cache

USER www-data

CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/supervisord.conf"]

Supervisor Configuration

; docker/supervisor/worker.conf
[supervisord]
nodaemon=true
logfile=/var/log/supervisor/supervisord.log
pidfile=/var/run/supervisord.pid

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/artisan queue:work redis \
    --queue=critical,high,default,low \
    --sleep=3 \
    --tries=3 \
    --max-time=3600 \
    --memory=256 \
    --timeout=90
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
numprocs=%(ENV_WORKER_NUMPROCS)s
redirect_stderr=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stopwaitsecs=3600

The WORKER_NUMPROCS environment variable lets you control the number of processes inside a container. Combined with Kubernetes, this provides two-level scaling: number of pods × number of processes per pod.

Deploying Workers as a Kubernetes Deployment with HPA

Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: laravel-queue-worker
  namespace: production
  labels:
    app: laravel-queue-worker
spec:
  replicas: 3
  selector:
    matchLabels:
      app: laravel-queue-worker
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2
      maxUnavailable: 0  # zero-downtime during updates
  template:
    metadata:
      labels:
        app: laravel-queue-worker
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9253"
    spec:
      terminationGracePeriodSeconds: 3600  # wait for long-running jobs to finish
      containers:
        - name: worker
          image: your-registry/laravel-app:latest
          imagePullPolicy: Always
          env:
            - name: APP_ENV
              value: production
            - name: QUEUE_CONNECTION
              value: redis
            - name: WORKER_NUMPROCS
              value: "2"
          envFrom:
            - secretRef:
                name: laravel-secrets
          resources:
            requests:
              memory: "256Mi"
              cpu: "200m"
            limits:
              memory: "512Mi"
              cpu: "500m"
          lifecycle:
            preStop:
              exec:
                # Graceful shutdown: wait for the current job to finish
                command: ["/bin/sh", "-c", "php artisan queue:restart && sleep 30"]
      restartPolicy: Always

Horizontal Pod Autoscaler (HPA)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: laravel-queue-worker-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: laravel-queue-worker
  minReplicas: 2
  maxReplicas: 20
  metrics:
    # Scale by CPU
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    # Scale by custom metric (queue size from Prometheus)
    - type: External
      external:
        metric:
          name: laravel_queue_size
          selector:
            matchLabels:
              queue: default
        target:
          type: AverageValue
          averageValue: "100"  # 1 pod per 100 jobs in the queue
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30
      policies:
        - type: Pods
          value: 4
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300  # scale down slowly to avoid flapping
      policies:
        - type: Pods
          value: 1
          periodSeconds: 120

Scaling based on a custom metric (Redis queue size) is the key advantage of this approach. When the queue grows, Kubernetes automatically adds workers; when it empties, it reduces their count, saving resources.

Queue Monitoring: Laravel Horizon and Prometheus

Setting Up Laravel Horizon

Laravel Horizon provides a beautiful dashboard and detailed queue statistics. Installation:

composer require laravel/horizon
php artisan horizon:install
php artisan migrate

config/horizon.php configuration for a Kubernetes environment:

<?php
// config/horizon.php

return [
    'use'     => 'default',
    'prefix'  => env('HORIZON_PREFIX', 'horizon:'),
    'middleware' => ['web', 'auth'],

    'waits' => [
        'redis:critical' => 3,   // alert if a job waits >3 sec
        'redis:default'  => 60,
        'redis:low'      => 300,
    ],

    'trim' => [
        'recent'        => 60,
        'pending'       => 60,
        'completed'     => 60,
        'recent_failed' => 10080, // 7 days
        'failed'        => 10080,
        'monitored'     => 10080,
    ],

    'silenced' => [],

    'metrics' => [
        'trim_snapshots' => [
            'job'   => 24,
            'queue' => 24,
        ],
    ],

    'fast_termination' => false,
    'memory_limit'     => 256,

    'environments' => [
        'production' => [
            'supervisor-critical' => [
                'connection' => 'redis',
                'queue'      => ['critical'],
                'balance'    => 'auto',
                'autoScalingStrategy' => 'time',
                'minProcesses' => 2,
                'maxProcesses' => 10,
                'tries'      => 3,
                'timeout'    => 90,
            ],
            'supervisor-default' => [
                'connection' => 'redis',
                'queue'      => ['high', 'default', 'low'],
                'balance'    => 'auto',
                'autoScalingStrategy' => 'size',
                'minProcesses' => 1,
                'maxProcesses' => 20,
                'balanceMaxShift'  => 3,
                'balanceCooldown' => 3,
                'tries'      => 3,
                'timeout'    => 120,
            ],
        ],
    ],
];

Prometheus Metrics for Kubernetes HPA

To export Laravel queue metrics to Prometheus, use the spatie/laravel-prometheus package or write your own endpoint:

<?php
// app/Http/Controllers/MetricsController.php

namespace App\\Http\\Controllers;

use Illuminate\\Support\\Facades\\Redis;

class MetricsController extends Controller
{
    public function index(): string
    {
        $queues = ['critical', 'high', 'default', 'low'];
        $output = '';

        foreach ($queues as $queue) {
            $size = Redis::connection('queue')->llen('queues:' . $queue);
            $output .= "laravel_queue_size{queue=\"$queue\"} $size\n";
        }

        // Failed jobs count
        $failedCount = Redis::connection('queue')->zcard('failed_jobs');
        $output .= "laravel_queue_failed_jobs_total $failedCount\n";

        return response($output, 200, ['Content-Type' => 'text/plain; version=0.0.4']);
    }
}

Register the route without middleware in routes/api.php:

Route::get('/metrics', [MetricsController::class, 'index'])
    ->middleware('throttle:60,1');

Handling Failed Jobs and Dead Letter Queue

Retry Configuration

In the job class, configure a retry strategy with exponential backoff:

<?php

class ProcessWebhook implements ShouldQueue
{
    public int $tries = 5;
    public int $timeout = 60;
    public int $maxExceptions = 3; // fail after 3 exceptions, even if tries are not exhausted

    // Exponential backoff: 10, 60, 180, 420 seconds
    public function backoff(): array
    {
        return [10, 60, 180, 420];
    }

    public function handle(): void
    {
        // process the webhook
    }

    public function failed(Throwable $exception): void
    {
        // Notify Slack/PagerDuty on final failure
        Log::error('Webhook processing failed', [
            'job_id'    => $this->job->getJobId(),
            'error'     => $exception->getMessage(),
            'payload'   => $this->webhook->toArray(),
        ]);

        // Push to dead letter queue for manual analysis
        DeadLetterQueue::push($this->webhook, $exception);
    }
}

Dead Letter Queue via a Dedicated Queue

Implement the Dead Letter Queue pattern using a named dead-letter queue and a separate monitoring worker:

<?php

class DeadLetterQueue
{
    public static function push(mixed $payload, Throwable $exception): void
    {
        Redis::connection('queue')->rpush('queues:dead-letter', json_encode([
            'payload'    => serialize($payload),
            'exception'  => $exception->getMessage(),
            'trace'      => $exception->getTraceAsString(),
            'failed_at'  => now()->toISOString(),
        ]));
    }

    public static function requeue(string $jobId): void
    {
        // Logic to re-dispatch from the DLQ
        Artisan::call('queue:retry', ['id' => [$jobId]]);
    }
}

Artisan Commands for Managing Failed Jobs

# List failed jobs
php artisan queue:failed

# Retry a specific job
php artisan queue:retry 5d5a4fb3-5e82-4af3-bedf-6df64b4ee59b

# Retry all failed jobs
php artisan queue:retry all

# Prune failed jobs older than 7 days
php artisan queue:prune-failed --hours=168

# Check Horizon status
php artisan horizon:status

Production Configuration Tips and Pitfalls

Graceful Shutdown in Kubernetes

One of the main pitfalls is the forced killing of a pod during scale-down. Set terminationGracePeriodSeconds equal to the maximum job timeout, and use a preStop hook to send the queue:restart signal. Workers will finish their current jobs and shut down cleanly.

Memory Leaks in Long-Running Workers

PHP does not free memory as efficiently as Go or Java. Use the --max-time and --memory flags so that workers restart before reaching critical thresholds. Supervisor will automatically start a new process.

Job Idempotency

Always design jobs to be idempotent: running the same job multiple times should not cause duplicate effects. Use Redis locks for critical sections:

<?php

public function handle(): void
{
    $lock = Cache::lock('process-order-' . $this->orderId, 120);

    if (! $lock->get()) {
        Log::warning('Job already running, skipping', ['order_id' => $this->orderId]);
        return;
    }

    try {
        $this->processOrder();
    } finally {
        $lock->release();
    }
}

Separating Queues by Criticality

Do not mix payment jobs and newsletter dispatches in the same queue. Allocate separate Deployments for critical queues with guaranteed resources and separate ones for background tasks with resource limits.

Monitoring Queue Lag

Set up alerts for queue wait time (queue lag). If a job in the critical queue has been waiting more than 5 seconds, that is already an incident. Laravel Horizon displays this metric out of the box on its dashboard.

Redis Persistence

Make sure Redis is configured with AOF persistence (appendonly yes) or use Redis Sentinel/Cluster. Losing queued data when Redis crashes without persistence is a real production risk.

Conclusion

The Laravel Queues + Redis Cluster + Kubernetes HPA stack in 2026 is a mature, production-ready approach to horizontal scaling of background jobs. You get automatic worker scaling based on actual queue size, graceful shutdown without job loss, detailed monitoring via Laravel Horizon and Prometheus, and reliable failure handling through a dead letter queue.

Key principles to remember: jobs should be idempotent, workers should be stateless and easily restartable, and monitoring queue metrics should be the foundation for autoscaling decisions. Start with the base configuration from this article and adapt it to the specifics of your workload.