PostgreSQL Table Partitioning: Strategies for Time-Series and Big Data Storage
Introduction: When Partitioning Helps and When It Hurts
Table partitioning in PostgreSQL is one of the most powerful tools for handling large volumes of data. However, it is not always the right choice. Before making a decision, it is important to understand the context.
Partitioning makes sense when:
A table contains hundreds of millions of rows or more, and queries always filter data by a range (time, region, category).
You need to regularly purge stale data — for example, retaining only the last 90 days of logs.
Different parts of the table have different access "temperatures": recent data is read frequently, older data rarely or never.
VACUUM and ANALYZE on a monolithic table take too long and block normal operations.
Partitioning is harmful when:
The table is small (up to 10–50 million rows under typical load) — the planner overhead will outweigh the benefit.
Queries do not filter by the partition key — the planner will scan all partitions (partition fan-out).
The application makes heavy use of ON CONFLICT (UPSERT) — this works with limitations on partitioned tables.
Global unique indexes that do not include the partition key are required.
In this article we will explore advanced partitioning strategies in PostgreSQL 15/16 for storing time-series data, event logs, and analytical data, walk through Laravel integration, and compare real-world performance results.
PostgreSQL Partitioning Types: RANGE, LIST, HASH
PostgreSQL supports three main declarative partitioning strategies, introduced in version 10 and significantly improved in versions 11–16.
RANGE — Range-Based Partitioning
The most common approach for time-series data. Each partition stores rows whose key value falls within a specific range.
CREATE TABLE events (
id BIGSERIAL,
occurred_at TIMESTAMPTZ NOT NULL,
user_id BIGINT,
event_type VARCHAR(64),
payload JSONB
) PARTITION BY RANGE (occurred_at);
Use cases for RANGE: time-series data (metrics, logs, transactions), data with a natural temporal or numeric progression, tables with date-based retention policies.
LIST — List-Based Partitioning
Each partition contains rows with specific key values from a predefined list.
CREATE TABLE orders (
id BIGSERIAL,
region VARCHAR(32) NOT NULL,
created_at TIMESTAMPTZ,
amount NUMERIC(12,2)
) PARTITION BY LIST (region);
CREATE TABLE orders_eu PARTITION OF orders FOR VALUES IN ('EU', 'UK', 'DE');
CREATE TABLE orders_us PARTITION OF orders FOR VALUES IN ('US', 'CA');
CREATE TABLE orders_apac PARTITION OF orders FOR VALUES IN ('JP', 'AU', 'SG');
Use cases for LIST: multi-tenant systems (partition per tenant), geographic sharding, data with a small number of discrete categories.
HASH — Hash-Based Partitioning
Rows are distributed evenly across partitions based on the hash of the key. Used when there is no natural range or list of values, but you need to distribute load horizontally.
CREATE TABLE user_activity (
user_id BIGINT NOT NULL,
activity_at TIMESTAMPTZ,
action VARCHAR(128)
) PARTITION BY HASH (user_id);
CREATE TABLE user_activity_0 PARTITION OF user_activity
FOR VALUES WITH (MODULUS 4, REMAINDER 0);
CREATE TABLE user_activity_1 PARTITION OF user_activity
FOR VALUES WITH (MODULUS 4, REMAINDER 1);
-- and so on
Strategy Comparison
RANGE: best choice for time-series, efficient partition pruning by date range, easy removal of old partitions.
LIST: efficient when filtering by category, but requires a known set of values in advance; scales poorly with a large number of unique values.
HASH: even data distribution, but partition pruning only works on exact key equality, and there is no straightforward archiving strategy.
Time-Based Partitioning: A Practical Example with an Events Table
Let's walk through a complete example of creating a partitioned events table with monthly partitions.
-- Create the parent table
CREATE TABLE events (
id BIGSERIAL,
occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
user_id BIGINT NOT NULL,
event_type VARCHAR(64) NOT NULL,
session_id UUID,
payload JSONB,
PRIMARY KEY (id, occurred_at)
) PARTITION BY RANGE (occurred_at);
-- Create partitions manually
CREATE TABLE events_2025_01 PARTITION OF events
FOR VALUES FROM ('2025-01-01') TO ('2025-02-01');
CREATE TABLE events_2025_02 PARTITION OF events
FOR VALUES FROM ('2025-02-01') TO ('2025-03-01');
CREATE TABLE events_2025_03 PARTITION OF events
FOR VALUES FROM ('2025-03-01') TO ('2025-04-01');
-- DEFAULT partition for data outside defined ranges
CREATE TABLE events_default PARTITION OF events DEFAULT;
Important note about PRIMARY KEY: in partitioned tables, the primary key must include the partition key. This is a PostgreSQL constraint stemming from the lack of global unique indexes.
Now let's create indexes on each partition (or on the parent table — starting with PostgreSQL 11, indexes on the parent are automatically propagated to child partitions):
-- Index on the parent table (PostgreSQL 11+)
CREATE INDEX idx_events_user_time ON events (user_id, occurred_at DESC);
CREATE INDEX idx_events_type ON events (event_type, occurred_at DESC);
Automatic Partition Creation: pg_partman and Manual Automation
pg_partman — Partition Management Extension
pg_partman is a PostgreSQL extension that automates partition creation and removal on a schedule. It supports RANGE partitioning by time and numeric ranges.
-- Install the extension
CREATE EXTENSION pg_partman SCHEMA partman;
-- Configure automatic partition management
SELECT partman.create_parent(
p_parent_table => 'public.events',
p_control => 'occurred_at',
p_type => 'native',
p_interval => 'monthly',
p_premake => 3 -- pre-create 3 future partitions
);
-- Update configuration
UPDATE partman.part_config
SET retention = '12 months',
retention_keep_table = false,
infinite_time_partitions = true
WHERE parent_table = 'public.events';
-- Run maintenance (typically via cron or pg_cron)
SELECT partman.run_maintenance();
pg_partman integrates with pg_cron for a fully automated partition lifecycle:
SELECT cron.schedule('partman-maintenance', '0 * * * *',
'SELECT partman.run_maintenance(p_analyze := false)');
Manual Automation via PL/pgSQL
If extension installation is restricted (e.g., managed PostgreSQL in the cloud), you can implement automation manually:
CREATE OR REPLACE FUNCTION create_monthly_partition(
p_table TEXT,
p_date DATE
) RETURNS VOID AS $$
DECLARE
partition_name TEXT;
start_date DATE;
end_date DATE;
BEGIN
start_date := DATE_TRUNC('month', p_date)::DATE;
end_date := (start_date + INTERVAL '1 month')::DATE;
partition_name := p_table || '_' || TO_CHAR(start_date, 'YYYY_MM');
EXECUTE FORMAT(
'CREATE TABLE IF NOT EXISTS %I PARTITION OF %I
FOR VALUES FROM (%L) TO (%L)',
partition_name, p_table, start_date, end_date
);
RAISE NOTICE 'Created partition: %', partition_name;
END;
$$ LANGUAGE plpgsql;
-- Create partitions for the next 3 months
SELECT create_monthly_partition('events', (NOW() + (i || ' months')::INTERVAL)::DATE)
FROM GENERATE_SERIES(0, 2) AS i;
Partition Pruning: How the PostgreSQL Planner Uses Partitions
Partition pruning is a mechanism by which the query planner excludes irrelevant partitions from the execution plan based on WHERE conditions. It is the key performance factor for partitioned tables.
PostgreSQL supports two levels of pruning:
Static pruning — partitions are excluded at planning time (when WHERE values are known at parse time).
Dynamic pruning — partitions are excluded at execution time (for parameterized queries, subplans, nested loops).
Verification with EXPLAIN ANALYZE
EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
SELECT COUNT(*), event_type
FROM events
WHERE occurred_at BETWEEN '2025-03-01' AND '2025-03-31'
GROUP BY event_type;
Example output with effective pruning:
HashAggregate (cost=8420.50..8421.00 rows=50 width=40)
(actual time=45.123..45.198 rows=47 loops=1)
Buffers: shared hit=3841
-> Append (cost=0.00..7980.00 rows=176000 width=32)
Subplans Removed: 11 -- <-- 11 partitions excluded!
-> Seq Scan on events_2025_03
(cost=0.00..3240.00 rows=176000 width=32)
Filter: ((occurred_at >= '2025-03-01') AND
(occurred_at < '2025-04-01'))
Planning Time: 2.341 ms
Execution Time: 45.891 ms
The line Subplans Removed: 11 means that 11 out of 12 partitions were excluded by the planner.
Key rules for partition pruning:
The WHERE condition must directly reference the partition key.
Do not wrap the key in functions:
DATE_TRUNC('month', occurred_at) = '2025-03-01'— pruning will not work. Use explicit ranges instead:occurred_at >= '2025-03-01' AND occurred_at < '2025-04-01'.The
enable_partition_pruningparameter must be enabled (ON by default in PostgreSQL 11+).
SHOW enable_partition_pruning; -- on
SET enable_partition_pruning = on;
Indexes on Partitioned Tables: Local vs Global, Partial Indexes
In PostgreSQL 11+, when an index is created on the parent table, it is automatically created on all child partitions. These are local indexes — each partition has its own B-tree.
Local Indexes
-- An index on the parent automatically creates indexes on all partitions
CREATE INDEX CONCURRENTLY idx_events_user_occurred
ON events (user_id, occurred_at DESC);
-- Check indexes on child tables
SELECT schemaname, tablename, indexname
FROM pg_indexes
WHERE tablename LIKE 'events_%'
ORDER BY tablename, indexname;
Global Index Limitations
Prior to PostgreSQL 17, global unique indexes spanning all partitions are not supported unless the uniqueness key includes the partition key. This is a fundamental constraint. PostgreSQL 17 is expected to support global indexes — keep an eye on release notes.
Partial Indexes on Partitions
Partial indexes are especially effective on partitioned tables — they index only a subset of rows, reducing index size and speeding up queries on "hot" data:
-- Partial index for error events only on a specific partition
CREATE INDEX idx_events_2025_03_errors
ON events_2025_03 (user_id, occurred_at)
WHERE event_type = 'error';
-- Partial index for pending transactions
CREATE INDEX idx_events_pending
ON events_2025_03 (occurred_at, user_id)
WHERE payload->>'status' = 'pending';
BRIN Indexes for Time-Series Data
For tables with correlated data (time-series where rows are physically ordered by time), BRIN indexes occupy minimal space while remaining effective:
CREATE INDEX idx_events_occurred_brin
ON events USING BRIN (occurred_at)
WITH (pages_per_range = 64);
A BRIN index on a 100-million-row table takes just a few megabytes compared to gigabytes for a B-tree index.
Archiving and Dropping Old Partitions: Retention Strategies
One of the biggest advantages of partitioning is the ability to instantly drop entire partitions instead of running slow row-by-row DELETEs.
Detach and Drop
-- Fast removal of an old partition (instant, minimal locking)
DROP TABLE events_2024_01;
-- Or: detach a partition without dropping it (for archiving)
ALTER TABLE events
DETACH PARTITION events_2024_01 CONCURRENTLY; -- PostgreSQL 14+
-- After detach, the table exists as a regular table
-- It can be moved to another tablespace, compressed, or dumped
ALTER TABLE events_2024_01 SET TABLESPACE archive_tablespace;
The key advantage: DROP TABLE on a partition completes in milliseconds regardless of row count, whereas DELETE FROM events WHERE occurred_at < '2024-02-01' on 100 million rows takes hours and generates enormous WAL output.
Automated Retention Strategy
CREATE OR REPLACE FUNCTION drop_old_partitions(
p_table TEXT,
p_retention_months INT DEFAULT 12
) RETURNS INT AS $$
DECLARE
rec RECORD;
dropped INT := 0;
cutoff DATE;
BEGIN
cutoff := DATE_TRUNC('month',
NOW() - (p_retention_months || ' months')::INTERVAL)::DATE;
FOR rec IN
SELECT child.relname AS partition_name
FROM pg_inherits
JOIN pg_class parent ON pg_inherits.inhparent = parent.oid
JOIN pg_class child ON pg_inherits.inhrelid = child.oid
WHERE parent.relname = p_table
AND child.relname ~ ('^' || p_table || '_\\d{4}_\\d{2}$')
LOOP
-- Extract date from partition name
IF TO_DATE(
REGEXP_REPLACE(rec.partition_name,
'^.*_(\\d{4})_(\\d{2})$', '\\1-\\2-01'),
'YYYY-MM-DD'
) < cutoff THEN
EXECUTE 'DROP TABLE ' || QUOTE_IDENT(rec.partition_name);
dropped := dropped + 1;
RAISE NOTICE 'Dropped partition: %', rec.partition_name;
END IF;
END LOOP;
RETURN dropped;
END;
$$ LANGUAGE plpgsql;
-- Usage
SELECT drop_old_partitions('events', 12); -- retain 12 months
Tablespace Strategy for Cold Data
Instead of dropping them, old partitions can be moved to slower (and cheaper) disks or object storage via extensions such as pg_tiering:
-- Create a tablespace on a slow disk / NFS
CREATE TABLESPACE cold_storage
LOCATION '/mnt/cold-data/pg';
-- Move a partition
ALTER TABLE events_2024_01 SET TABLESPACE cold_storage;
Laravel Integration: Working with Partitioned Tables
Laravel and Eloquent work well with PostgreSQL partitioned tables — from the ORM's perspective, the table looks just like a regular one. There are, however, a few nuances to keep in mind.
Migrations
Laravel's Schema Builder does not support the PARTITION BY syntax, so migrations must be written using raw SQL:
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;
return new class extends Migration
{
public function up(): void
{
// Create the partitioned table
DB::statement('CREATE TABLE events (
id BIGSERIAL,
occurred_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
user_id BIGINT NOT NULL,
event_type VARCHAR(64) NOT NULL,
payload JSONB,
PRIMARY KEY (id, occurred_at)
) PARTITION BY RANGE (occurred_at)');
// Create initial partitions
DB::statement("CREATE TABLE events_2025_01
PARTITION OF events
FOR VALUES FROM ('2025-01-01') TO ('2025-02-01')");
DB::statement("CREATE TABLE events_default
PARTITION OF events DEFAULT");
// Create indexes
DB::statement('CREATE INDEX idx_events_user_time
ON events (user_id, occurred_at DESC)');
}
public function down(): void
{
DB::statement('DROP TABLE IF EXISTS events CASCADE');
}
};
Eloquent Model
<?php
namespace App\Models;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Builder;
class Event extends Model
{
protected $table = 'events';
protected $primaryKey = 'id';
public $timestamps = false;
protected $casts = [
'occurred_at' => 'datetime',
'payload' => 'array',
];
protected $fillable = [
'occurred_at', 'user_id', 'event_type', 'session_id', 'payload',
];
// Scope for effective partition pruning
public function scopeInPeriod(Builder $query, string $from, string $to): Builder
{
// Important: use whereBetween or explicit operators,
// do NOT wrap in functions like DATE_TRUNC
return $query
->where('occurred_at', '>=', $from)
->where('occurred_at', '<', $to);
}
public function scopeForUser(Builder $query, int $userId): Builder
{
return $query->where('user_id', $userId);
}
}
Efficient Queries via Eloquent
<?php
// Good query — partition pruning will work
$events = Event::inPeriod('2025-03-01', '2025-04-01')
->forUser($userId)
->select(['id', 'occurred_at', 'event_type'])
->orderBy('occurred_at', 'desc')
->limit(100)
->get();
// Aggregation with pruning
$stats = Event::inPeriod('2025-03-01', '2025-04-01')
->selectRaw('event_type, COUNT(*) as cnt, DATE(occurred_at) as day')
->groupBy('event_type', 'day')
->orderBy('day', 'desc')
->get();
// Raw query for complex analytics
$result = DB::select("
SELECT
DATE_TRUNC('hour', occurred_at) AS hour,
event_type,
COUNT(*) AS count,
COUNT(DISTINCT user_id) AS unique_users
FROM events
WHERE occurred_at >= ? AND occurred_at < ?
AND event_type = ANY(?)
GROUP BY 1, 2
ORDER BY 1 DESC
", [
'2025-03-01',
'2025-04-01',
'{purchase,signup,error}'
]);
Caching Results with Redis
For analytical queries on partitioned tables, using Redis as a result cache is highly effective — especially for historical data that no longer changes:
<?php
use Illuminate\Support\Facades\Cache;
public function getHourlyStats(string $date): array
{
$cacheKey = "events:hourly:{$date}";
// Cache historical data for a long time
$ttl = Carbon::parse($date)->isPast() ? 86400 * 7 : 300;
return Cache::store('redis')->remember($cacheKey, $ttl, function () use ($date) {
return DB::select("
SELECT DATE_TRUNC('hour', occurred_at) AS hour,
COUNT(*) AS total
FROM events
WHERE occurred_at >= ?::DATE
AND occurred_at < (?::DATE + INTERVAL '1 day')
GROUP BY 1 ORDER BY 1
", [$date, $date]);
});
}
Redis is particularly effective for "cold" partitions: once computed, aggregates are cached for days, completely offloading PostgreSQL.
Benchmarks: Real Numbers Before and After Partitioning
Below are test results on an events table with 500 million rows (PostgreSQL 16, 32 CPUs, 128 GB RAM, NVMe SSD, shared_buffers = 32 GB).
Test 1: Aggregation Over One Month (Out of 24 Months of Data)
SELECT event_type, COUNT(*)
FROM events
WHERE occurred_at BETWEEN '2025-03-01' AND '2025-03-31'
GROUP BY event_type;
Without partitioning: 47.3 seconds (Seq Scan, 500M rows)
With RANGE partitioning (monthly): 1.2 seconds (Seq Scan on events_2025_03 only, ~21M rows)
With partitioning + B-tree index: 0.18 seconds (Index Scan)
Speedup: ~260×
Test 2: INSERT Performance
-- Batch insert of 1 million rows
INSERT INTO events (occurred_at, user_id, event_type, payload)
SELECT
NOW() - (RANDOM() * INTERVAL '30 days'),
(RANDOM() * 1000000)::BIGINT,
(ARRAY['click','view','purchase','error'])[CEIL(RANDOM()*4)::INT],
'{"v": 1}'::JSONB
FROM GENERATE_SERIES(1, 1000000);
Without partitioning: 8.4 seconds
With partitioning (data into 1 partition): 9.1 seconds (+8% overhead)
With partitioning (data spread across 30 partitions): 11.3 seconds (+35% overhead)
Takeaway: partitioning slightly slows down INSERTs due to row routing overhead. For high-throughput systems, consider writing directly to a specific partition or using COPY.
Test 3: Deleting One Month of Data
DELETE FROM events WHERE occurred_at < '2024-02-01': 38 minutes, 12 GB of WAL
DROP TABLE events_2024_01: 0.003 seconds, minimal WAL
Speedup: >760,000×
Test 4: Index Sizes
B-tree on monolithic table (500M rows): 18.4 GB
Total B-tree across 24 partitions: 18.9 GB (nearly identical)
BRIN on monolithic table: 47 MB
BRIN on partitioned table: 52 MB
BRIN indexes provide enormous storage savings with minimal performance loss for sequential time-range queries.
Conclusion
Table partitioning in PostgreSQL is not a silver bullet — it is a surgical tool. Used correctly, it delivers dramatic query speedups through partition pruning, instant removal of stale data, more efficient VACUUM operation, and fine-grained storage control.
Key takeaways:
Use RANGE partitioning by time for logs, metrics, and event data — this is the most common and well-optimized scenario in PostgreSQL.
Always include the partition key in critical queries; otherwise the planner will perform a full scan of all partitions.
pg_partman + pg_cron is the de facto standard for automating partition lifecycle management in production.
BRIN indexes on time-series data save orders of magnitude more space than B-tree while delivering comparable performance for range queries.
Laravel integration is transparent — use raw migrations and Eloquent scopes that explicitly pass partition key conditions.
Redis perfectly complements partitioned tables for caching aggregates over historical, immutable partitions.
As data volumes continue to grow in 2026, well-implemented PostgreSQL partitioning remains one of the most cost-effective solutions available — before moving to more complex architectures (sharding, Citus, TimescaleDB), make sure you have fully exhausted the capabilities of native partitioning.
Technologies
Tags
Ruslan Ismailov
Senior Web / Backend Developer. Senior web/backend developer with 9 years of experience. Stack: PHP, Laravel, PostgreSQL, Redis, Docker, Kubernetes, REST, microservices, CI/CD. More about me →