Case study
A real-time analytics platform
Collecting and processing an event stream with real-time dashboards on Go, PostgreSQL and Redis.
- Year
- 2025
- Client
- SaaS analytics platform
- Role
- Tech Lead
Description
The analytics platform ingests a stream of user and system events, processes it on the fly and delivers aggregated metrics to dashboards with minimal latency. The event ingestion and processing service is written in Go for high throughput and predictable latency: goroutines serve thousands of simultaneous connections, events are buffered and written to storage in batches. Hot aggregates (counters, sliding windows, tops) are computed in Redis using sorted sets and HyperLogLog, while historical data is stored in PostgreSQL with window functions for analytical queries. The product layer and the dashboard-management API are implemented in Laravel. The whole system is containerized and deployed in Kubernetes with horizontal autoscaling of the processors under load, which lets it survive peak traffic spikes without losing events.
Task
The existing analytics was computed by nightly batch jobs, data lagged by a day, and as traffic grew processing could not keep up and lost events. The business needed up-to-date metrics in real time.
Solution
I built an event-ingestion service in Go with batch writing, moved hot aggregates into Redis, organized a historical store in PostgreSQL with window functions, and deployed the processors in Kubernetes with autoscaling.
Results
The delay from event to dashboard dropped from 24 hours to 3 seconds, event loss at peaks was eliminated, the platform steadily processes over 50,000 events per second, and infrastructure costs were reduced by 35%.