Contract-First & Schema Evolution

Serialization formats at boundaries

Format	Schema	Human-readable	Typical use
JSON	Optional (JSON Schema)	Yes	Public REST, config
XML	XSD/DTD	Verbose	Legacy enterprise, SOAP
Protocol Buffers	.proto required	No (binary)	gRPC, internal RPC
Avro	.avsc with registry	No	Kafka, data pipelines
MessagePack/CBOR	Informal	No	Compact JSON-like

Topic 5 goes deep on protobuf/gRPC. Here: evolution rules apply to all.

Compatibility modes (Confluent/Protobuf registry)

• Backward — new schema reads old data (add field with default)
• Forward — old schema reads new data (add optional field only)
• Full — both directions
• Breaking — rename/remove without alias → new subject version or topic

Rules of thumb:

• Add optional fields — usually safe
• Remove fields — breaking unless consumers ignore unknowns
• Change type — almost always breaking
• Enum — never reuse numeric values

Consumer-driven contracts (Pact)

Consumer defines expected request/response; provider verifies in CI. Catches breaking changes before deploy — critical when 30 teams consume your API.

Flow: consumer test generates pact file → published to broker → provider verification job fails if response shape drifts.

Contract-first workflow

1. Design OpenAPI / .proto / AsyncAPI in PR
2. Review with consumers
3. Generate stubs
4. Implement + contract tests
5. Deploy with compatibility checks in CI

Versioning serialized messages

For async (Kafka): new topic orders-v2 or schema registry version with compatibility check.

For sync (REST): URL/header version + additive JSON fields.

Dual-write / dual-read migration window when breaking change unavoidable — staff-level migration pattern.

JSON Schema at HTTP boundaries

Validate request bodies at gateway or service edge. OpenAPI 3 embeds JSON Schema. Catches bugs early; watch validation latency on hot paths.

Cross-reference: Data Engineering for Avro in Kafka pipelines; Databases → Object Storage for how Parquet is stored — this topic owns the HTTP/API contract and consumer-driven tests at team boundaries.