Backend development

Data Validation and Transformation in Go: Building a Reliable Input Layer for REST APIs Without Frameworks

Ruslan Ismailov Published 14 min read
D

Introduction: Why Validation Is More Than Just Checking Fields

Validating incoming data is one of those tasks developers underestimate until the first serious incident. It seems straightforward: check that email is not empty and contains @. But in practice, a poorly designed validation layer destroys the architecture: business logic gets mixed with HTTP parsing, domain objects accept invalid states, and errors are returned to clients in different formats depending on which developer wrote a particular handler.

In the Go ecosystem, it is common to build REST APIs without heavy frameworks — and rightly so. The standard library's net/http provides everything you need. But in this context, architectural decisions around validation must be made independently. In this article, we will explore how to build a reliable, testable, and maintainable validation and data transformation layer for a REST API in Go.

Overview of Validation Approaches in Go

Manual Validation

The simplest approach is writing checks manually inside handler or method bodies. Pros: full control, no dependencies, easy to read. Cons: code duplication, difficulty collecting all errors in a single pass, and rules scattered across the codebase.

Struct Tags

A popular approach in the Go world: validation rules are written as struct field tags. Convenient for simple cases, but tags quickly become unreadable, do not support complex conditional logic, and bind validation rules to the data structure, violating separation of concerns.

The go-playground/validator Library

The most popular solution in the ecosystem. Supports tags, custom validators, and localization. Pros: rich set of built-in rules, active community. Cons: reflection under the hood affects performance, the API can be non-obvious, and custom rules registered via RegisterValidation are difficult to test in isolation. For high-load services, this trade-off is worth considering carefully.

In this article, we implement a lightweight custom validation layer, showing where it can be replaced or supplemented with go-playground/validator if needed.

Designing the Validation Layer: Where Rules Live

The key architectural question is: where should validation live? The answer depends on the type of check:

  • Syntactic validation (email format, string length, required fields) — handler or middleware level. This is the "entry gate" — structurally incorrect data should never pass through.
  • Semantic validation (email uniqueness in the database, business constraints) — usecase/service level. This is where repositories and business context are accessible.
  • Domain model invariants (e.g., an order total cannot be negative) — domain level, in constructors or factory methods.

Mixing these levels is the primary reason validation breaks architecture. A handler should not know about business rules, and a domain object should not parse HTTP requests.

Implementing a Custom Validator

Let's define the interface and basic primitives. The idea: the validator collects all errors in a single pass and returns them as a list, rather than stopping at the first error.

// validation/validator.go
package validation

import (
	"fmt"
	"net/mail"
	"regexp"
	"strings"
)

// FieldError describes an error for a specific field.
type FieldError struct {
	Field   string `json:"field"`
	Message string `json:"message"`
	Code    string `json:"code"`
}

func (e FieldError) Error() string {
	return fmt.Sprintf("%s: %s", e.Field, e.Message)
}

// ValidationErrors is a list of validation errors.
type ValidationErrors []FieldError

func (ve ValidationErrors) Error() string {
	msgs := make([]string, len(ve))
	for i, e := range ve {
		msgs[i] = e.Error()
	}
	return strings.Join(msgs, "; ")
}

// Rule is a rule function: takes a field name and value, returns an error or nil.
type Rule func(field, value string) *FieldError

// FieldValidator accumulates rules for a single field.
type FieldValidator struct {
	field string
	value string
	rules []Rule
}

// Validator is the root object containing rule sets for all fields.
type Validator struct {
	fields []*FieldValidator
}

// Field starts a rule chain for a field.
func (v *Validator) Field(field, value string) *FieldValidator {
	fv := &FieldValidator{field: field, value: value}
	v.fields = append(v.fields, fv)
	return fv
}

// Required checks that the field is not empty.
func (fv *FieldValidator) Required() *FieldValidator {
	fv.rules = append(fv.rules, func(field, value string) *FieldError {
		if strings.TrimSpace(value) == "" {
			return &FieldError{Field: field, Message: "field is required", Code: "required"}
		}
		return nil
	})
	return fv
}

// MinLen checks the minimum string length.
func (fv *FieldValidator) MinLen(min int) *FieldValidator {
	fv.rules = append(fv.rules, func(field, value string) *FieldError {
		if len([]rune(value)) < min {
			return &FieldError{
				Field:   field,
				Message: fmt.Sprintf("minimum length: %d characters", min),
				Code:    "min_length",
			}
		}
		return nil
	})
	return fv
}

// MaxLen checks the maximum string length.
func (fv *FieldValidator) MaxLen(max int) *FieldValidator {
	fv.rules = append(fv.rules, func(field, value string) *FieldError {
		if len([]rune(value)) > max {
			return &FieldError{
				Field:   field,
				Message: fmt.Sprintf("maximum length: %d characters", max),
				Code:    "max_length",
			}
		}
		return nil
	})
	return fv
}

// Email checks the format of an email address.
func (fv *FieldValidator) Email() *FieldValidator {
	fv.rules = append(fv.rules, func(field, value string) *FieldError {
		_, err := mail.ParseAddress(value)
		if err != nil {
			return &FieldError{Field: field, Message: "invalid email address", Code: "invalid_email"}
		}
		return nil
	})
	return fv
}

// Matches checks that the value matches a regular expression.
func (fv *FieldValidator) Matches(pattern, code, message string) *FieldValidator {
	re := regexp.MustCompile(pattern)
	fv.rules = append(fv.rules, func(field, value string) *FieldError {
		if !re.MatchString(value) {
			return &FieldError{Field: field, Message: message, Code: code}
		}
		return nil
	})
	return fv
}

// Validate runs all rules and returns a list of errors.
func (v *Validator) Validate() ValidationErrors {
	var errs ValidationErrors
	for _, fv := range v.fields {
		for _, rule := range fv.rules {
			if err := rule(fv.field, fv.value); err != nil {
				errs = append(errs, *err)
				// Continue — collect all errors for the field
			}
		}
	}
	if len(errs) == 0 {
		return nil
	}
	return errs
}

This design provides a fluent API that is easy to read and test. Each rule is a pure function with no side effects.

Data Transformation: The DTO Pattern and Mapping to Domain Objects

A DTO (Data Transfer Object) is a structure that describes the shape of an incoming request. It is not a domain object and contains no business logic. After validation, the DTO is transformed into a domain model.

// dto/user.go
package dto

import (
	"strings"
	"time"

	"myapp/domain"
	"myapp/validation"
)

// CreateUserRequest is the DTO for an incoming user creation request.
type CreateUserRequest struct {
	Name     string `json:"name"`
	Email    string `json:"email"`
	Password string `json:"password"`
	Role     string `json:"role"`
}

// Validate performs syntactic validation of the DTO.
func (r *CreateUserRequest) Validate() validation.ValidationErrors {
	v := &validation.Validator{}
	v.Field("name", r.Name).Required().MinLen(2).MaxLen(100)
	v.Field("email", r.Email).Required().Email()
	v.Field("password", r.Password).Required().MinLen(8).MaxLen(72)
	v.Field("role", r.Role).Required().Matches(
		`^(admin|user|moderator)$`,
		"invalid_role",
		"allowed roles: admin, user, moderator",
	)
	return v.Validate()
}

// ToDomain transforms the DTO into a domain object.
// Called only after successful validation.
func (r *CreateUserRequest) ToDomain() domain.CreateUserParams {
	return domain.CreateUserParams{
		Name:      strings.TrimSpace(r.Name),
		Email:     strings.ToLower(strings.TrimSpace(r.Email)),
		Password:  r.Password, // hashing happens at the usecase level
		Role:      domain.Role(r.Role),
		CreatedAt: time.Now().UTC(),
	}
}

An important principle: the ToDomain() method is called only after successful validation. It may perform data normalization — strings.TrimSpace, lowercasing, type conversion — but not business logic.

Handling and Returning Validation Errors: RFC 7807 Problem Details

RFC 7807 defines a standard error response format for HTTP APIs. Using this standard makes the API predictable for clients and compatible with OpenAPI documentation.

// apierror/problem.go
package apierror

import (
	"encoding/json"
	"net/http"

	"myapp/validation"
)

// ProblemDetail conforms to RFC 7807.
type ProblemDetail struct {
	Type     string       `json:"type"`
	Title    string       `json:"title"`
	Status   int          `json:"status"`
	Detail   string       `json:"detail,omitempty"`
	Instance string       `json:"instance,omitempty"`
	Errors   []FieldError `json:"errors,omitempty"`
}

// FieldError is a field in the error response.
type FieldError struct {
	Field   string `json:"field"`
	Message string `json:"message"`
	Code    string `json:"code"`
}

// WriteValidationError writes a response containing validation errors.
func WriteValidationError(w http.ResponseWriter, r *http.Request, errs validation.ValidationErrors) {
	fields := make([]FieldError, len(errs))
	for i, e := range errs {
		fields[i] = FieldError{Field: e.Field, Message: e.Message, Code: e.Code}
	}

	problem := ProblemDetail{
		Type:     "https://api.example.com/errors/validation",
		Title:    "Validation Error",
		Status:   http.StatusUnprocessableEntity,
		Detail:   "The submitted data contains errors.",
		Instance: r.RequestURI,
		Errors:   fields,
	}

	w.Header().Set("Content-Type", "application/problem+json")
	w.WriteHeader(http.StatusUnprocessableEntity)
	json.NewEncoder(w).Encode(problem)
}

// WriteInternalError writes a response for an internal server error.
func WriteInternalError(w http.ResponseWriter, r *http.Request) {
	problem := ProblemDetail{
		Type:     "https://api.example.com/errors/internal",
		Title:    "Internal Server Error",
		Status:   http.StatusInternalServerError,
		Instance: r.RequestURI,
	}
	w.Header().Set("Content-Type", "application/problem+json")
	w.WriteHeader(http.StatusInternalServerError)
	json.NewEncoder(w).Encode(problem)
}

Example response to a client upon validation errors:

HTTP/1.1 422 Unprocessable Entity
Content-Type: application/problem+json

{
  "type": "https://api.example.com/errors/validation",
  "title": "Validation Error",
  "status": 422,
  "detail": "The submitted data contains errors.",
  "instance": "/api/v1/users",
  "errors": [
    {"field": "email", "message": "invalid email address", "code": "invalid_email"},
    {"field": "password", "message": "minimum length: 8 characters", "code": "min_length"}
  ]
}

Localizing Error Messages

If the API serves multiple locales, error messages should be returned in the client's language. A simple approach is mapping error codes to localized strings.

// validation/i18n.go
package validation

type Locale string

const (
	LocaleRU Locale = "ru"
	LocaleEN Locale = "en"
)

// messages holds translations by error code and locale.
var messages = map[Locale]map[string]string{
	LocaleRU: {
		"required":      "обязательное поле",
		"min_length":    "значение слишком короткое",
		"max_length":    "значение слишком длинное",
		"invalid_email": "некорректный адрес электронной почты",
	},
	LocaleEN: {
		"required":      "field is required",
		"min_length":    "value is too short",
		"max_length":    "value is too long",
		"invalid_email": "invalid email address",
	},
}

// Translate returns a localized message for the given error code.
func Translate(locale Locale, code string) string {
	if msgs, ok := messages[locale]; ok {
		if msg, ok := msgs[code]; ok {
			return msg
		}
	}
	// Fall back to English
	if msg, ok := messages[LocaleEN][code]; ok {
		return msg
	}
	return code
}

The locale is determined from the Accept-Language header and passed to the handler via the request context.

Testing the Validation Layer

One of the main advantages of this approach is testability. Validators are pure functions with no dependencies. We use table-driven tests:

// validation/validator_test.go
package validation_test

import (
	"testing"

	"myapp/validation"
)

func TestEmailValidation(t *testing.T) {
	tests := []struct {
		name     string
		email    string
		wantErr  bool
		wantCode string
	}{
		{name: "valid email", email: "user@example.com", wantErr: false},
		{name: "empty email", email: "", wantErr: true, wantCode: "required"},
		{name: "no at sign", email: "notanemail", wantErr: true, wantCode: "invalid_email"},
		{name: "no domain", email: "user@", wantErr: true, wantCode: "invalid_email"},
		{name: "with spaces", email: "user @example.com", wantErr: true, wantCode: "invalid_email"},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			v := &validation.Validator{}
			v.Field("email", tt.email).Required().Email()
			errs := v.Validate()

			if tt.wantErr && len(errs) == 0 {
				t.Errorf("expected an error, but validation passed")
			}
			if !tt.wantErr && len(errs) > 0 {
				t.Errorf("did not expect an error, but got: %v", errs)
			}
			if tt.wantErr && len(errs) > 0 && tt.wantCode != "" {
				found := false
				for _, e := range errs {
					if e.Code == tt.wantCode {
						found = true
					}
				}
				if !found {
					t.Errorf("expected code %q, but it was not found in errors: %v", tt.wantCode, errs)
				}
			}
		})
	}
}

// TestCreateUserRequestValidate tests the full DTO validation.
func TestCreateUserRequestValidate(t *testing.T) {
	tests := []struct {
		name       string
		input      dto.CreateUserRequest
		wantFields []string // fields expected to have errors
	}{
		{
			name: "valid input",
			input: dto.CreateUserRequest{
				Name: "Ivan Petrov", Email: "ivan@example.com",
				Password: "securePass1", Role: "user",
			},
			wantFields: nil,
		},
		{
			name: "missing name and invalid role",
			input: dto.CreateUserRequest{
				Name: "", Email: "ivan@example.com",
				Password: "securePass1", Role: "superadmin",
			},
			wantFields: []string{"name", "role"},
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			errs := tt.input.Validate()
			errFields := map[string]bool{}
			for _, e := range errs {
				errFields[e.Field] = true
			}
			for _, f := range tt.wantFields {
				if !errFields[f] {
					t.Errorf("expected an error in field %q, but none was found", f)
				}
			}
			if len(tt.wantFields) == 0 && len(errs) > 0 {
				t.Errorf("did not expect errors, but got: %v", errs)
			}
		})
	}
}

Middleware Integration: Automatic Validation at the Router Level

To avoid duplicating parsing and validation code in every handler, we extract this logic into middleware. We use generics (Go 1.18+) for type-safe decoding.

// middleware/validate.go
package middleware

import (
	"context"
	"encoding/json"
	"net/http"

	"myapp/apierror"
	"myapp/validation"
)

// Validatable is an interface for DTOs with a validation method.
type Validatable interface {
	Validate() validation.ValidationErrors
}

// contextKey is a type for context keys to avoid collisions.
type contextKey[T any] struct{}

// DecodeAndValidate is a middleware factory for decoding and validating the request body.
// Uses generics: T must implement Validatable.
func DecodeAndValidate[T Validatable](next func(http.ResponseWriter, *http.Request, T)) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		var dto T

		// Limit request body size (DoS protection)
		r.Body = http.MaxBytesReader(w, r.Body, 1<<20) // 1 MB

		decoder := json.NewDecoder(r.Body)
		decoder.DisallowUnknownFields() // strict mode

		if err := decoder.Decode(&dto); err != nil {
			apierror.WriteBadRequestError(w, r, "invalid JSON")
			return
		}

		if errs := dto.Validate(); errs != nil {
			apierror.WriteValidationError(w, r, errs)
			return
		}

		next(w, r, dto)
	}
}

Example usage in a handler:

// handler/user.go
package handler

import (
	"net/http"

	"myapp/dto"
	"myapp/middleware"
	"myapp/usecase"
)

type UserHandler struct {
	userUC usecase.UserUseCase
}

func (h *UserHandler) Register(mux *http.ServeMux) {
	// middleware.DecodeAndValidate handles decoding and validation
	mux.HandleFunc("POST /api/v1/users",
		middleware.DecodeAndValidate(h.createUser),
	)
}

func (h *UserHandler) createUser(w http.ResponseWriter, r *http.Request, req dto.CreateUserRequest) {
	// req is already valid here — safe to transform
	params := req.ToDomain()

	user, err := h.userUC.CreateUser(r.Context(), params)
	if err != nil {
		// Handle business errors
		return
	}

	w.WriteHeader(http.StatusCreated)
	// ... serialize response
	_ = user
}

This approach completely eliminates boilerplate from handlers. The handler receives an already-validated DTO and focuses solely on its own responsibility — orchestrating business logic.

Conclusion: A Checklist for a Good Validation Layer

To summarize, here are the key characteristics of a high-quality validation layer in a Go REST API:

  • Separation of concerns: syntactic validation belongs in the handler/middleware, semantic validation in the usecase, and invariants in the domain.
  • All errors in a single pass: the client should receive a complete list of issues rather than fixing them one at a time.
  • DTO is separate from the domain object: transformation happens explicitly via the ToDomain() method, and only after validation.
  • Standardized error format: RFC 7807 makes the API predictable for all consumers.
  • Localization via error codes: do not hardcode strings in rules — use codes and translate them separately.
  • 100% test coverage for validators: every rule is tested in isolation using a table-driven approach.
  • Middleware eliminates boilerplate: handlers do not deal with parsing or validation — only with logic.
  • Request body size limit: http.MaxBytesReader is mandatory as a minimum protection measure.
  • Minimal dependencies: the standard library covers most needs; add go-playground/validator only when genuinely necessary.

The validation layer built this way scales easily: adding a new rule is a single function of type Rule, and adding a new DTO means implementing the Validatable interface. The architecture stays clean, tests run fast, and the API remains predictable for clients.

Technologies

Tags

Ruslan Ismailov

Senior Web / Backend Developer. Senior web/backend developer with 9 years of experience. Stack: PHP, Laravel, PostgreSQL, Redis, Docker, Kubernetes, REST, microservices, CI/CD. More about me →