Kubernetes Operators in Go: Automating Custom Resource Management in 2026
Introduction: What Is a Kubernetes Operator and Why Do You Need One
Kubernetes provides a powerful declarative API for infrastructure management, but standard abstractions — Deployment, StatefulSet, Service — are often insufficient for managing complex stateful applications. Helm charts solve the problem of manifest templating but cannot react to cluster events or make real-time decisions. Kubernetes Operators fill this gap by encoding operational knowledge (runbooks) directly into a controller running inside the cluster.
An Operator is a Kubernetes extension pattern consisting of two parts: a Custom Resource Definition (CRD) that describes the desired state, and a controller that reconciles the actual cluster state with the desired one. Unlike Helm, an Operator can perform scheduled backups, automatically recover a PostgreSQL cluster after a failure, rotate secrets, and execute rolling upgrades with custom business logic.
By 2026, the Operator pattern has become the de facto standard for platform teams: OperatorHub.io lists over 400 ready-made solutions, and the Go toolchain — kubebuilder and controller-runtime — has reached a high level of maturity. In this article, we'll walk through everything from core concepts to a working Operator for managing PostgreSQL instances.
Key Concepts: CRD, Controller, and Reconciliation Loop
Custom Resource Definition
A CRD is an extension of the Kubernetes API schema. Once a CRD manifest is applied, the cluster starts accepting objects of the new kind — for example, PostgreSQLCluster. Users interact with it via kubectl just as they would with a Deployment.
apiVersion: apiextensions.k8s.io/v1\nkind: CustomResourceDefinition\nmetadata:\n name: postgresqlclusters.db.example.com\nspec:\n group: db.example.com\n versions:\n - name: v1alpha1\n served: true\n storage: true\n schema:\n openAPIV3Schema:\n type: object\n properties:\n spec:\n type: object\n properties:\n replicas:\n type: integer\n minimum: 1\n version:\n type: string\n status:\n type: object\n properties:\n phase:\n type: string\n scope: Namespaced\n names:\n plural: postgresqlclusters\n singular: postgresqlcluster\n kind: PostgreSQLCluster\nReconciliation Loop
The heart of any Operator is the reconciliation loop. The controller subscribes to events (object creation, modification, deletion) and calls the Reconcile function on each event. The function fetches the current state from the cluster and decides what actions are needed to bring the actual state in line with the desired state described in spec. The most important property of a reconciler is idempotency: calling it repeatedly with the same state must not produce any side effects.
"Don't think about events — think about state. A reconciler always answers one question: what needs to happen right now to reach the desired state?"
Tooling: kubebuilder vs operator-sdk in 2026
The two main scaffolding tools for Go operators are kubebuilder (maintained by sig-controller-tools) and operator-sdk (Red Hat). In 2026, the difference between them is minimal: operator-sdk uses kubebuilder as its foundation and adds OLM (Operator Lifecycle Manager) integration along with plugins for Ansible/Helm operators.
- kubebuilder — the choice for teams that want minimal overhead and full control over the codebase.
- operator-sdk — the choice for teams planning to publish on OperatorHub or working within the OpenShift ecosystem.
- Both use controller-runtime — a Go library that abstracts the low-level details of working with the Kubernetes API.
In this article, we use kubebuilder v4, which is current as of 2026.
Step-by-Step Operator Creation in Go
Step 1: Project Scaffolding
Install kubebuilder and initialize the project:
# Install kubebuilder\ncurl -L -o kubebuilder https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)\nchmod +x kubebuilder && mv kubebuilder /usr/local/bin/\n\n# Initialize the project\nmkdir postgres-operator && cd postgres-operator\nkubebuilder init --domain example.com --repo github.com/example/postgres-operator\n\n# Create API and controller\nkubebuilder create api --group db --version v1alpha1 --kind PostgreSQLCluster\nAfter running these commands, kubebuilder generates the project structure: api/v1alpha1/ contains Go types for the CRD, internal/controller/ holds the reconciler scaffold, and config/ contains kustomize manifests.
Step 2: Defining CRD Types in Go
// api/v1alpha1/postgresqlcluster_types.go
package v1alpha1
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
// PostgreSQLClusterSpec describes the desired state of the cluster
type PostgreSQLClusterSpec struct {
// Replicas — number of replicas (1 primary + N standbys)
Replicas int32 `json:"replicas"`
// Version — PostgreSQL version, e.g. "16.2"
Version string `json:"version"`
// StorageSize — PVC size for each instance
StorageSize string `json:"storageSize"`
// BackupSchedule — cron schedule for backups
BackupSchedule string `json:"backupSchedule,omitempty"`
}
// PostgreSQLClusterStatus reflects the actual state
type PostgreSQLClusterStatus struct {
Phase string `json:"phase,omitempty"`
ReadyNodes int32 `json:"readyNodes,omitempty"`
Conditions []metav1.Condition `json:"conditions,omitempty"`
}
// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:printcolumn:name="Phase",type="string",JSONPath=".status.phase"
// +kubebuilder:printcolumn:name="Ready",type="integer",JSONPath=".status.readyNodes"
type PostgreSQLCluster struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec PostgreSQLClusterSpec `json:"spec,omitempty"`
Status PostgreSQLClusterStatus `json:"status,omitempty"`
}
// +kubebuilder:object:root=true
type PostgreSQLClusterList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []PostgreSQLCluster `json:"items"`
}
func init() {
SchemeBuilder.Register(&PostgreSQLCluster{}, &PostgreSQLClusterList{})
}
After modifying the types, run make generate manifests — controller-gen will automatically update the CRD YAML and generate DeepCopy methods.
Step 3: Implementing the Reconciler
// internal/controller/postgresqlcluster_controller.go
package controller
import (
"context"
"fmt"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/log"
dbv1alpha1 "github.com/example/postgres-operator/api/v1alpha1"
)
type PostgreSQLClusterReconciler struct {
client.Client
Scheme *runtime.Scheme
}
// +kubebuilder:rbac:groups=db.example.com,resources=postgresqlclusters,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=db.example.com,resources=postgresqlclusters/status,verbs=get;update;patch
// +kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete
func (r *PostgreSQLClusterReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
logger := log.FromContext(ctx)
// 1. Fetch the object from the cluster
cluster := &dbv1alpha1.PostgreSQLCluster{}
if err := r.Get(ctx, req.NamespacedName, cluster); err != nil {
if errors.IsNotFound(err) {
// Object deleted — nothing to do
return ctrl.Result{}, nil
}
return ctrl.Result{}, fmt.Errorf("failed to get PostgreSQLCluster: %w", err)
}
logger.Info("Reconciling", "name", cluster.Name, "phase", cluster.Status.Phase)
// 2. Ensure the StatefulSet exists and matches the spec
if err := r.reconcileStatefulSet(ctx, cluster); err != nil {
return ctrl.Result{}, err
}
// 3. Ensure the Service exists
if err := r.reconcileService(ctx, cluster); err != nil {
return ctrl.Result{}, err
}
// 4. Update status
if err := r.updateStatus(ctx, cluster); err != nil {
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
func (r *PostgreSQLClusterReconciler) reconcileStatefulSet(
ctx context.Context,
cluster *dbv1alpha1.PostgreSQLCluster,
) error {
desired := r.buildStatefulSet(cluster)
// Set owner reference for garbage collection
if err := ctrl.SetControllerReference(cluster, desired, r.Scheme); err != nil {
return err
}
existing := &appsv1.StatefulSet{}
err := r.Get(ctx, client.ObjectKeyFromObject(desired), existing)
if errors.IsNotFound(err) {
return r.Create(ctx, desired)
}
if err != nil {
return err
}
// Update only if replicas or image changed
existing.Spec.Replicas = desired.Spec.Replicas
existing.Spec.Template = desired.Spec.Template
return r.Update(ctx, existing)
}
func (r *PostgreSQLClusterReconciler) buildStatefulSet(
cluster *dbv1alpha1.PostgreSQLCluster,
) *appsv1.StatefulSet {
image := fmt.Sprintf("postgres:%s", cluster.Spec.Version)
replicas := cluster.Spec.Replicas
return &appsv1.StatefulSet{
ObjectMeta: metav1.ObjectMeta{
Name: cluster.Name,
Namespace: cluster.Namespace,
},
Spec: appsv1.StatefulSetSpec{
Replicas: &replicas,
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{"app": cluster.Name},
},
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{"app": cluster.Name},
},
Spec: corev1.PodSpec{
Containers: []corev1.Container{
{
Name: "postgres",
Image: image,
Env: []corev1.EnvVar{
{
Name: "POSTGRES_PASSWORD",
Value: "changeme", // in production, use a Secret
},
},
},
},
},
},
},
}
}
func (r *PostgreSQLClusterReconciler) reconcileService(
ctx context.Context,
cluster *dbv1alpha1.PostgreSQLCluster,
) error {
svc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: cluster.Name + "-svc",
Namespace: cluster.Namespace,
},
Spec: corev1.ServiceSpec{
Selector: map[string]string{"app": cluster.Name},
Ports: []corev1.ServicePort{
{Port: 5432},
},
},
}
if err := ctrl.SetControllerReference(cluster, svc, r.Scheme); err != nil {
return err
}
existing := &corev1.Service{}
if err := r.Get(ctx, client.ObjectKeyFromObject(svc), existing); errors.IsNotFound(err) {
return r.Create(ctx, svc)
} else {
return err
}
}
func (r *PostgreSQLClusterReconciler) updateStatus(
ctx context.Context,
cluster *dbv1alpha1.PostgreSQLCluster,
) error {
cluster.Status.Phase = "Running"
cluster.Status.ReadyNodes = cluster.Spec.Replicas
return r.Status().Update(ctx, cluster)
}
func (r *PostgreSQLClusterReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&dbv1alpha1.PostgreSQLCluster{}).
Owns(&appsv1.StatefulSet{}).
Owns(&corev1.Service{}).
Complete(r)
}
Practical Example: PostgreSQL Backup and Restore
Let's extend the Operator with backup functionality. We'll add a PostgreSQLBackup CRD and a reconciler that creates a Kubernetes Job to run pg_dump.
// Backup reconciler fragment
func (r *PostgreSQLBackupReconciler) Reconcile(
ctx context.Context,
req ctrl.Request,
) (ctrl.Result, error) {
backup := &dbv1alpha1.PostgreSQLBackup{}
if err := r.Get(ctx, req.NamespacedName, backup); err != nil {
return ctrl.Result{}, client.IgnoreNotFound(err)
}
// Idempotency: skip if Job already exists
job := &batchv1.Job{}
jobName := fmt.Sprintf("%s-backup-job", backup.Name)
err := r.Get(ctx, types.NamespacedName{
Name: jobName,
Namespace: backup.Namespace,
}, job)
if errors.IsNotFound(err) {
newJob := r.buildBackupJob(backup, jobName)
if err := ctrl.SetControllerReference(backup, newJob, r.Scheme); err != nil {
return ctrl.Result{}, err
}
if err := r.Create(ctx, newJob); err != nil {
return ctrl.Result{}, fmt.Errorf("failed to create backup job: %w", err)
}
backup.Status.Phase = "Running"
return ctrl.Result{RequeueAfter: 30 * time.Second},
r.Status().Update(ctx, backup)
}
// Check Job result
if job.Status.Succeeded > 0 {
backup.Status.Phase = "Completed"
backup.Status.CompletedAt = &metav1.Time{Time: time.Now()}
} else if job.Status.Failed > 3 {
backup.Status.Phase = "Failed"
} else {
// Job still running
return ctrl.Result{RequeueAfter: 15 * time.Second}, nil
}
return ctrl.Result{}, r.Status().Update(ctx, backup)
}
func (r *PostgreSQLBackupReconciler) buildBackupJob(
backup *dbv1alpha1.PostgreSQLBackup,
name string,
) *batchv1.Job {
clusterSvc := backup.Spec.ClusterName + "-svc"
return &batchv1.Job{
ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: backup.Namespace},
Spec: batchv1.JobSpec{
Template: corev1.PodTemplateSpec{
Spec: corev1.PodSpec{
RestartPolicy: corev1.RestartPolicyOnFailure,
Containers: []corev1.Container{
{
Name: "pg-dump",
Image: "postgres:16",
Command: []string{
"pg_dump",
"-h", clusterSvc,
"-U", "postgres",
"-F", "c",
"-f", "/backup/dump.pgc",
backup.Spec.Database,
},
},
},
},
},
},
}
}
Error Handling and Idempotency
The reconciliation loop must be resilient to network failures, version conflicts (optimistic locking), and repeated invocations. Key rules:
- Always re-fetch the object at the start of Reconcile — don't rely on the cached state from the event.
- Use
ctrl.Result{RequeueAfter: ...}for deferred checks instead of a blocking sleep. - Wrap errors using
fmt.Errorf("...: %w", err)for proper error tracing. - Handle conflicts: when
errors.IsConflict(err)occurs, simply return the error — the controller will automatically retry the reconcile. - Use finalizers for cleanup logic when an object is deleted: add the finalizer on creation, remove it after cleanup is complete.
// Finalizer usage example
const finalizer = "db.example.com/cleanup"
if cluster.DeletionTimestamp.IsZero() {
// Object is not being deleted — add finalizer
if !controllerutil.ContainsFinalizer(cluster, finalizer) {
controllerutil.AddFinalizer(cluster, finalizer)
return ctrl.Result{}, r.Update(ctx, cluster)
}
} else {
// Object is being deleted — perform cleanup
if controllerutil.ContainsFinalizer(cluster, finalizer) {
if err := r.cleanupExternalResources(ctx, cluster); err != nil {
return ctrl.Result{}, err
}
controllerutil.RemoveFinalizer(cluster, finalizer)
return ctrl.Result{}, r.Update(ctx, cluster)
}
}
Testing the Operator: envtest and Integration Tests
controller-runtime ships with the envtest package, which runs a real Kubernetes API server and etcd locally — without a full cluster. This enables writing integration tests that verify the complete reconciliation cycle.
// internal/controller/suite_test.go
package controller_test
import (
"testing"
"path/filepath"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"sigs.k8s.io/controller-runtime/pkg/envtest"
"sigs.k8s.io/controller-runtime/pkg/client"
)
var (
testEnv *envtest.Environment
k8sClient client.Client
)
func TestControllers(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "Controller Suite")
}
var _ = BeforeSuite(func() {
testEnv = &envtest.Environment{
CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")},
}
cfg, err := testEnv.Start()
Expect(err).NotTo(HaveOccurred())
k8sClient, err = client.New(cfg, client.Options{})
Expect(err).NotTo(HaveOccurred())
})
var _ = AfterSuite(func() {
Expect(testEnv.Stop()).To(Succeed())
})
// internal/controller/postgresqlcluster_controller_test.go
var _ = Describe("PostgreSQLCluster Controller", func() {
It("should create a StatefulSet for a new cluster", func() {
ctx := context.Background()
cluster := &dbv1alpha1.PostgreSQLCluster{
ObjectMeta: metav1.ObjectMeta{
Name: "test-cluster",
Namespace: "default",
},
Spec: dbv1alpha1.PostgreSQLClusterSpec{
Replicas: 2,
Version: "16.2",
StorageSize: "10Gi",
},
}
Expect(k8sClient.Create(ctx, cluster)).To(Succeed())
sts := &appsv1.StatefulSet{}
Eventually(func() error {
return k8sClient.Get(ctx, types.NamespacedName{
Name: "test-cluster",
Namespace: "default",
}, sts)
}, "10s", "1s").Should(Succeed())
Expect(*sts.Spec.Replicas).To(Equal(int32(2)))
})
})
Deploying the Operator: kustomize, Helm, and OLM
kubebuilder generates a ready-to-use config/ directory with kustomize manifests. Deployment is straightforward:
# Build and push the image
make docker-build docker-push IMG=ghcr.io/example/postgres-operator:v0.1.0
# Deploy with kustomize
make deploy IMG=ghcr.io/example/postgres-operator:v0.1.0
# Or directly
kubectl apply -k config/default
For production deployments, it is recommended to package the Operator as a Helm chart with configurable parameters via values.yaml. If you plan to publish on OperatorHub, use operator-sdk to generate an OLM bundle:
operator-sdk generate bundle \
--package postgres-operator \
--version 0.1.0 \
--channels stable
operator-sdk bundle validate ./bundle
Security: RBAC and the Principle of Least Privilege
The Operator runs with the permissions of a ServiceAccount in the cluster. The // +kubebuilder:rbac: annotations in the controller code automatically generate a ClusterRole when you run make manifests. Follow the principle of least privilege:
- Grant access only to the resources and namespaces that are actually needed.
- Where possible, use a namespace-scoped Role instead of a ClusterRole.
- Never grant the Operator
cluster-adminprivileges. - Use Admission Webhooks (ValidatingWebhookConfiguration) to validate CRDs at the API server level.
- Store secrets (database passwords, S3 keys) in Kubernetes Secrets or External Secrets Operator — not in the CRD
spec.
# Example of a generated ClusterRole (excerpt)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: postgres-operator-manager-role
rules:
- apiGroups: ["db.example.com"]
resources: ["postgresqlclusters"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["db.example.com"]
resources: ["postgresqlclusters/status"]
verbs: ["get", "update", "patch"]
- apiGroups: ["apps"]
resources: ["statefulsets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
resources: ["services", "persistentvolumeclaims"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
Conclusion: When to Write an Operator vs. Using Standard Tools
A Kubernetes Operator is a powerful tool, but not a silver bullet. Here are the criteria for making the right decision:
- Write an Operator when you need to automate operational procedures (backup/restore, failover, rolling upgrades with health checks) that require real-time reaction to cluster events.
- Write an Operator when you have a complex stateful service (database, message queue, broker) with non-trivial scaling logic.
- Use Helm or kustomize when the task comes down to manifest templating and configuration management without complex business logic.
- Use an existing Operator (CloudNativePG for PostgreSQL, Strimzi for Kafka) if it meets your requirements — don't reinvent the wheel.
In 2026, the Kubernetes Operators ecosystem in Go has reached maturity: kubebuilder v4 + controller-runtime provide a solid foundation, and patterns like idempotent reconciliation and envtest-based testing have become industry standards. For platform engineers and Go developers, mastering this stack unlocks the ability to build truly self-managing infrastructure.
Technologies
Tags
Ruslan Ismailov
Senior Web / Backend Developer. Senior web/backend developer with 9 years of experience. Stack: PHP, Laravel, PostgreSQL, Redis, Docker, Kubernetes, REST, microservices, CI/CD. More about me →