Databases

PostgreSQL Table Partitioning: Strategies for Time-Series and Big Data Storage

Ruslan Ismailov Published 18 min read
P

Introduction: When Partitioning Helps and When It Hurts

Table partitioning in PostgreSQL is one of the most powerful tools for handling large volumes of data. However, it is not always the right choice. Before making a decision, it is important to understand the context.

Partitioning makes sense when:

  • A table contains hundreds of millions of rows or more, and queries always filter data by a range (time, region, category).

  • You need to regularly purge stale data — for example, retaining only the last 90 days of logs.

  • Different parts of the table have different access "temperatures": recent data is read frequently, older data rarely or never.

  • VACUUM and ANALYZE on a monolithic table take too long and block normal operations.

Partitioning is harmful when:

  • The table is small (up to 10–50 million rows under typical load) — the planner overhead will outweigh the benefit.

  • Queries do not filter by the partition key — the planner will scan all partitions (partition fan-out).

  • The application makes heavy use of ON CONFLICT (UPSERT) — this works with limitations on partitioned tables.

  • Global unique indexes that do not include the partition key are required.

In this article we will explore advanced partitioning strategies in PostgreSQL 15/16 for storing time-series data, event logs, and analytical data, walk through Laravel integration, and compare real-world performance results.

PostgreSQL Partitioning Types: RANGE, LIST, HASH

PostgreSQL supports three main declarative partitioning strategies, introduced in version 10 and significantly improved in versions 11–16.

RANGE — Range-Based Partitioning

The most common approach for time-series data. Each partition stores rows whose key value falls within a specific range.

CREATE TABLE events (
    id BIGSERIAL,
    occurred_at TIMESTAMPTZ NOT NULL,
    user_id BIGINT,
    event_type VARCHAR(64),
    payload JSONB
) PARTITION BY RANGE (occurred_at);

Use cases for RANGE: time-series data (metrics, logs, transactions), data with a natural temporal or numeric progression, tables with date-based retention policies.

LIST — List-Based Partitioning

Each partition contains rows with specific key values from a predefined list.

CREATE TABLE orders (
    id BIGSERIAL,
    region VARCHAR(32) NOT NULL,
    created_at TIMESTAMPTZ,
    amount NUMERIC(12,2)
) PARTITION BY LIST (region);

CREATE TABLE orders_eu PARTITION OF orders FOR VALUES IN ('EU', 'UK', 'DE');
CREATE TABLE orders_us PARTITION OF orders FOR VALUES IN ('US', 'CA');
CREATE TABLE orders_apac PARTITION OF orders FOR VALUES IN ('JP', 'AU', 'SG');

Use cases for LIST: multi-tenant systems (partition per tenant), geographic sharding, data with a small number of discrete categories.

HASH — Hash-Based Partitioning

Rows are distributed evenly across partitions based on the hash of the key. Used when there is no natural range or list of values, but you need to distribute load horizontally.

CREATE TABLE user_activity (
    user_id BIGINT NOT NULL,
    activity_at TIMESTAMPTZ,
    action VARCHAR(128)
) PARTITION BY HASH (user_id);

CREATE TABLE user_activity_0 PARTITION OF user_activity
    FOR VALUES WITH (MODULUS 4, REMAINDER 0);
CREATE TABLE user_activity_1 PARTITION OF user_activity
    FOR VALUES WITH (MODULUS 4, REMAINDER 1);
-- and so on

Strategy Comparison

  • RANGE: best choice for time-series, efficient partition pruning by date range, easy removal of old partitions.

  • LIST: efficient when filtering by category, but requires a known set of values in advance; scales poorly with a large number of unique values.

  • HASH: even data distribution, but partition pruning only works on exact key equality, and there is no straightforward archiving strategy.

Time-Based Partitioning: A Practical Example with an Events Table

Let's walk through a complete example of creating a partitioned events table with monthly partitions.

-- Create the parent table
CREATE TABLE events (
    id BIGSERIAL,
    occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    user_id BIGINT NOT NULL,
    event_type VARCHAR(64) NOT NULL,
    session_id UUID,
    payload JSONB,
    PRIMARY KEY (id, occurred_at)
) PARTITION BY RANGE (occurred_at);

-- Create partitions manually
CREATE TABLE events_2025_01 PARTITION OF events
    FOR VALUES FROM ('2025-01-01') TO ('2025-02-01');

CREATE TABLE events_2025_02 PARTITION OF events
    FOR VALUES FROM ('2025-02-01') TO ('2025-03-01');

CREATE TABLE events_2025_03 PARTITION OF events
    FOR VALUES FROM ('2025-03-01') TO ('2025-04-01');

-- DEFAULT partition for data outside defined ranges
CREATE TABLE events_default PARTITION OF events DEFAULT;

Important note about PRIMARY KEY: in partitioned tables, the primary key must include the partition key. This is a PostgreSQL constraint stemming from the lack of global unique indexes.

Now let's create indexes on each partition (or on the parent table — starting with PostgreSQL 11, indexes on the parent are automatically propagated to child partitions):

-- Index on the parent table (PostgreSQL 11+)
CREATE INDEX idx_events_user_time ON events (user_id, occurred_at DESC);
CREATE INDEX idx_events_type ON events (event_type, occurred_at DESC);

Automatic Partition Creation: pg_partman and Manual Automation

pg_partman — Partition Management Extension

pg_partman is a PostgreSQL extension that automates partition creation and removal on a schedule. It supports RANGE partitioning by time and numeric ranges.

-- Install the extension
CREATE EXTENSION pg_partman SCHEMA partman;

-- Configure automatic partition management
SELECT partman.create_parent(
    p_parent_table => 'public.events',
    p_control => 'occurred_at',
    p_type => 'native',
    p_interval => 'monthly',
    p_premake => 3 -- pre-create 3 future partitions
);

-- Update configuration
UPDATE partman.part_config
SET retention = '12 months',
    retention_keep_table = false,
    infinite_time_partitions = true
WHERE parent_table = 'public.events';

-- Run maintenance (typically via cron or pg_cron)
SELECT partman.run_maintenance();

pg_partman integrates with pg_cron for a fully automated partition lifecycle:

SELECT cron.schedule('partman-maintenance', '0 * * * *',
    'SELECT partman.run_maintenance(p_analyze := false)');

Manual Automation via PL/pgSQL

If extension installation is restricted (e.g., managed PostgreSQL in the cloud), you can implement automation manually:

CREATE OR REPLACE FUNCTION create_monthly_partition(
    p_table TEXT,
    p_date DATE
) RETURNS VOID AS $$
DECLARE
    partition_name TEXT;
    start_date DATE;
    end_date DATE;
BEGIN
    start_date := DATE_TRUNC('month', p_date)::DATE;
    end_date := (start_date + INTERVAL '1 month')::DATE;
    partition_name := p_table || '_' || TO_CHAR(start_date, 'YYYY_MM');

    EXECUTE FORMAT(
        'CREATE TABLE IF NOT EXISTS %I PARTITION OF %I
         FOR VALUES FROM (%L) TO (%L)',
        partition_name, p_table, start_date, end_date
    );

    RAISE NOTICE 'Created partition: %', partition_name;
END;
$$ LANGUAGE plpgsql;

-- Create partitions for the next 3 months
SELECT create_monthly_partition('events', (NOW() + (i || ' months')::INTERVAL)::DATE)
FROM GENERATE_SERIES(0, 2) AS i;

Partition Pruning: How the PostgreSQL Planner Uses Partitions

Partition pruning is a mechanism by which the query planner excludes irrelevant partitions from the execution plan based on WHERE conditions. It is the key performance factor for partitioned tables.

PostgreSQL supports two levels of pruning:

  • Static pruning — partitions are excluded at planning time (when WHERE values are known at parse time).

  • Dynamic pruning — partitions are excluded at execution time (for parameterized queries, subplans, nested loops).

Verification with EXPLAIN ANALYZE

EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
SELECT COUNT(*), event_type
FROM events
WHERE occurred_at BETWEEN '2025-03-01' AND '2025-03-31'
GROUP BY event_type;

Example output with effective pruning:

HashAggregate  (cost=8420.50..8421.00 rows=50 width=40)
               (actual time=45.123..45.198 rows=47 loops=1)
  Buffers: shared hit=3841
  ->  Append  (cost=0.00..7980.00 rows=176000 width=32)
        Subplans Removed: 11  -- <-- 11 partitions excluded!
        ->  Seq Scan on events_2025_03
              (cost=0.00..3240.00 rows=176000 width=32)
              Filter: ((occurred_at >= '2025-03-01') AND
                       (occurred_at < '2025-04-01'))
Planning Time: 2.341 ms
Execution Time: 45.891 ms

The line Subplans Removed: 11 means that 11 out of 12 partitions were excluded by the planner.

Key rules for partition pruning:

  • The WHERE condition must directly reference the partition key.

  • Do not wrap the key in functions: DATE_TRUNC('month', occurred_at) = '2025-03-01' — pruning will not work. Use explicit ranges instead: occurred_at >= '2025-03-01' AND occurred_at < '2025-04-01'.

  • The enable_partition_pruning parameter must be enabled (ON by default in PostgreSQL 11+).

SHOW enable_partition_pruning; -- on
SET enable_partition_pruning = on;

Indexes on Partitioned Tables: Local vs Global, Partial Indexes

In PostgreSQL 11+, when an index is created on the parent table, it is automatically created on all child partitions. These are local indexes — each partition has its own B-tree.

Local Indexes

-- An index on the parent automatically creates indexes on all partitions
CREATE INDEX CONCURRENTLY idx_events_user_occurred
ON events (user_id, occurred_at DESC);

-- Check indexes on child tables
SELECT schemaname, tablename, indexname
FROM pg_indexes
WHERE tablename LIKE 'events_%'
ORDER BY tablename, indexname;

Global Index Limitations

Prior to PostgreSQL 17, global unique indexes spanning all partitions are not supported unless the uniqueness key includes the partition key. This is a fundamental constraint. PostgreSQL 17 is expected to support global indexes — keep an eye on release notes.

Partial Indexes on Partitions

Partial indexes are especially effective on partitioned tables — they index only a subset of rows, reducing index size and speeding up queries on "hot" data:

-- Partial index for error events only on a specific partition
CREATE INDEX idx_events_2025_03_errors
ON events_2025_03 (user_id, occurred_at)
WHERE event_type = 'error';

-- Partial index for pending transactions
CREATE INDEX idx_events_pending
ON events_2025_03 (occurred_at, user_id)
WHERE payload->>'status' = 'pending';

BRIN Indexes for Time-Series Data

For tables with correlated data (time-series where rows are physically ordered by time), BRIN indexes occupy minimal space while remaining effective:

CREATE INDEX idx_events_occurred_brin
ON events USING BRIN (occurred_at)
WITH (pages_per_range = 64);

A BRIN index on a 100-million-row table takes just a few megabytes compared to gigabytes for a B-tree index.

Archiving and Dropping Old Partitions: Retention Strategies

One of the biggest advantages of partitioning is the ability to instantly drop entire partitions instead of running slow row-by-row DELETEs.

Detach and Drop

-- Fast removal of an old partition (instant, minimal locking)
DROP TABLE events_2024_01;

-- Or: detach a partition without dropping it (for archiving)
ALTER TABLE events
    DETACH PARTITION events_2024_01 CONCURRENTLY; -- PostgreSQL 14+

-- After detach, the table exists as a regular table
-- It can be moved to another tablespace, compressed, or dumped
ALTER TABLE events_2024_01 SET TABLESPACE archive_tablespace;

The key advantage: DROP TABLE on a partition completes in milliseconds regardless of row count, whereas DELETE FROM events WHERE occurred_at < '2024-02-01' on 100 million rows takes hours and generates enormous WAL output.

Automated Retention Strategy

CREATE OR REPLACE FUNCTION drop_old_partitions(
    p_table TEXT,
    p_retention_months INT DEFAULT 12
) RETURNS INT AS $$
DECLARE
    rec RECORD;
    dropped INT := 0;
    cutoff DATE;
BEGIN
    cutoff := DATE_TRUNC('month',
        NOW() - (p_retention_months || ' months')::INTERVAL)::DATE;

    FOR rec IN
        SELECT child.relname AS partition_name
        FROM pg_inherits
        JOIN pg_class parent ON pg_inherits.inhparent = parent.oid
        JOIN pg_class child  ON pg_inherits.inhrelid  = child.oid
        WHERE parent.relname = p_table
          AND child.relname ~ ('^' || p_table || '_\\d{4}_\\d{2}$')
    LOOP
        -- Extract date from partition name
        IF TO_DATE(
            REGEXP_REPLACE(rec.partition_name,
                '^.*_(\\d{4})_(\\d{2})$', '\\1-\\2-01'),
            'YYYY-MM-DD'
        ) < cutoff THEN
            EXECUTE 'DROP TABLE ' || QUOTE_IDENT(rec.partition_name);
            dropped := dropped + 1;
            RAISE NOTICE 'Dropped partition: %', rec.partition_name;
        END IF;
    END LOOP;

    RETURN dropped;
END;
$$ LANGUAGE plpgsql;

-- Usage
SELECT drop_old_partitions('events', 12); -- retain 12 months

Tablespace Strategy for Cold Data

Instead of dropping them, old partitions can be moved to slower (and cheaper) disks or object storage via extensions such as pg_tiering:

-- Create a tablespace on a slow disk / NFS
CREATE TABLESPACE cold_storage
    LOCATION '/mnt/cold-data/pg';

-- Move a partition
ALTER TABLE events_2024_01 SET TABLESPACE cold_storage;

Laravel Integration: Working with Partitioned Tables

Laravel and Eloquent work well with PostgreSQL partitioned tables — from the ORM's perspective, the table looks just like a regular one. There are, however, a few nuances to keep in mind.

Migrations

Laravel's Schema Builder does not support the PARTITION BY syntax, so migrations must be written using raw SQL:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;

return new class extends Migration
{
    public function up(): void
    {
        // Create the partitioned table
        DB::statement('CREATE TABLE events (
            id BIGSERIAL,
            occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
            user_id BIGINT NOT NULL,
            event_type VARCHAR(64) NOT NULL,
            payload JSONB,
            PRIMARY KEY (id, occurred_at)
        ) PARTITION BY RANGE (occurred_at)');

        // Create initial partitions
        DB::statement("CREATE TABLE events_2025_01
            PARTITION OF events
            FOR VALUES FROM ('2025-01-01') TO ('2025-02-01')");

        DB::statement("CREATE TABLE events_default
            PARTITION OF events DEFAULT");

        // Create indexes
        DB::statement('CREATE INDEX idx_events_user_time
            ON events (user_id, occurred_at DESC)');
    }

    public function down(): void
    {
        DB::statement('DROP TABLE IF EXISTS events CASCADE');
    }
};

Eloquent Model

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Builder;

class Event extends Model
{
    protected $table = 'events';
    protected $primaryKey = 'id';
    public $timestamps = false;

    protected $casts = [
        'occurred_at' => 'datetime',
        'payload'     => 'array',
    ];

    protected $fillable = [
        'occurred_at', 'user_id', 'event_type', 'session_id', 'payload',
    ];

    // Scope for effective partition pruning
    public function scopeInPeriod(Builder $query, string $from, string $to): Builder
    {
        // Important: use whereBetween or explicit operators,
        // do NOT wrap in functions like DATE_TRUNC
        return $query
            ->where('occurred_at', '>=', $from)
            ->where('occurred_at', '<', $to);
    }

    public function scopeForUser(Builder $query, int $userId): Builder
    {
        return $query->where('user_id', $userId);
    }
}

Efficient Queries via Eloquent

<?php

// Good query — partition pruning will work
$events = Event::inPeriod('2025-03-01', '2025-04-01')
    ->forUser($userId)
    ->select(['id', 'occurred_at', 'event_type'])
    ->orderBy('occurred_at', 'desc')
    ->limit(100)
    ->get();

// Aggregation with pruning
$stats = Event::inPeriod('2025-03-01', '2025-04-01')
    ->selectRaw('event_type, COUNT(*) as cnt, DATE(occurred_at) as day')
    ->groupBy('event_type', 'day')
    ->orderBy('day', 'desc')
    ->get();

// Raw query for complex analytics
$result = DB::select("
    SELECT
        DATE_TRUNC('hour', occurred_at) AS hour,
        event_type,
        COUNT(*) AS count,
        COUNT(DISTINCT user_id) AS unique_users
    FROM events
    WHERE occurred_at >= ? AND occurred_at < ?
      AND event_type = ANY(?)
    GROUP BY 1, 2
    ORDER BY 1 DESC
", [
    '2025-03-01',
    '2025-04-01',
    '{purchase,signup,error}'
]);

Caching Results with Redis

For analytical queries on partitioned tables, using Redis as a result cache is highly effective — especially for historical data that no longer changes:

<?php

use Illuminate\Support\Facades\Cache;

public function getHourlyStats(string $date): array
{
    $cacheKey = "events:hourly:{$date}";

    // Cache historical data for a long time
    $ttl = Carbon::parse($date)->isPast() ? 86400 * 7 : 300;

    return Cache::store('redis')->remember($cacheKey, $ttl, function () use ($date) {
        return DB::select("
            SELECT DATE_TRUNC('hour', occurred_at) AS hour,
                   COUNT(*) AS total
            FROM events
            WHERE occurred_at >= ?::DATE
              AND occurred_at < (?::DATE + INTERVAL '1 day')
            GROUP BY 1 ORDER BY 1
        ", [$date, $date]);
    });
}

Redis is particularly effective for "cold" partitions: once computed, aggregates are cached for days, completely offloading PostgreSQL.

Benchmarks: Real Numbers Before and After Partitioning

Below are test results on an events table with 500 million rows (PostgreSQL 16, 32 CPUs, 128 GB RAM, NVMe SSD, shared_buffers = 32 GB).

Test 1: Aggregation Over One Month (Out of 24 Months of Data)

SELECT event_type, COUNT(*)
FROM events
WHERE occurred_at BETWEEN '2025-03-01' AND '2025-03-31'
GROUP BY event_type;
  • Without partitioning: 47.3 seconds (Seq Scan, 500M rows)

  • With RANGE partitioning (monthly): 1.2 seconds (Seq Scan on events_2025_03 only, ~21M rows)

  • With partitioning + B-tree index: 0.18 seconds (Index Scan)

  • Speedup: ~260×

Test 2: INSERT Performance

-- Batch insert of 1 million rows
INSERT INTO events (occurred_at, user_id, event_type, payload)
SELECT
    NOW() - (RANDOM() * INTERVAL '30 days'),
    (RANDOM() * 1000000)::BIGINT,
    (ARRAY['click','view','purchase','error'])[CEIL(RANDOM()*4)::INT],
    '{"v": 1}'::JSONB
FROM GENERATE_SERIES(1, 1000000);
  • Without partitioning: 8.4 seconds

  • With partitioning (data into 1 partition): 9.1 seconds (+8% overhead)

  • With partitioning (data spread across 30 partitions): 11.3 seconds (+35% overhead)

Takeaway: partitioning slightly slows down INSERTs due to row routing overhead. For high-throughput systems, consider writing directly to a specific partition or using COPY.

Test 3: Deleting One Month of Data

  • DELETE FROM events WHERE occurred_at < '2024-02-01': 38 minutes, 12 GB of WAL

  • DROP TABLE events_2024_01: 0.003 seconds, minimal WAL

  • Speedup: >760,000×

Test 4: Index Sizes

  • B-tree on monolithic table (500M rows): 18.4 GB

  • Total B-tree across 24 partitions: 18.9 GB (nearly identical)

  • BRIN on monolithic table: 47 MB

  • BRIN on partitioned table: 52 MB

BRIN indexes provide enormous storage savings with minimal performance loss for sequential time-range queries.

Conclusion

Table partitioning in PostgreSQL is not a silver bullet — it is a surgical tool. Used correctly, it delivers dramatic query speedups through partition pruning, instant removal of stale data, more efficient VACUUM operation, and fine-grained storage control.

Key takeaways:

  • Use RANGE partitioning by time for logs, metrics, and event data — this is the most common and well-optimized scenario in PostgreSQL.

  • Always include the partition key in critical queries; otherwise the planner will perform a full scan of all partitions.

  • pg_partman + pg_cron is the de facto standard for automating partition lifecycle management in production.

  • BRIN indexes on time-series data save orders of magnitude more space than B-tree while delivering comparable performance for range queries.

  • Laravel integration is transparent — use raw migrations and Eloquent scopes that explicitly pass partition key conditions.

  • Redis perfectly complements partitioned tables for caching aggregates over historical, immutable partitions.

As data volumes continue to grow in 2026, well-implemented PostgreSQL partitioning remains one of the most cost-effective solutions available — before moving to more complex architectures (sharding, Citus, TimescaleDB), make sure you have fully exhausted the capabilities of native partitioning.

Technologies

Tags

Ruslan Ismailov

Senior Web / Backend Developer. Senior web/backend developer with 9 years of experience. Stack: PHP, Laravel, PostgreSQL, Redis, Docker, Kubernetes, REST, microservices, CI/CD. More about me →