Phased Rollout Plan

System: Integration Platform (IIP) Related docs: Architecture · Use Cases · Data Model · Implementation Plan · Generalization Strategy

This document takes each Release's backlog (Implementation Plan §4) and breaks it into much smaller, sequential, individually-completable phases — small enough that each one is roughly a single sitting's work and a single commit. It's an execution checklist, not an architecture artifact: check items off as they land, and expect the phase breakdown for later releases to be revised once earlier ones are actually built (that's normal XP feedback, not drift).
Releases 1–6 are complete. The current line is Release 7, phase 7.1 — and per the re-sequencing in 06 §3, Release 3 was contract extraction rather than schema-registry plumbing. Releases 4–9 are broken down to the same granularity so the whole rollout is visible up front, but per the project's own "simple design" / YAGNI practice (Implementation Plan §1), don't start building Release 6's control-plane UI while Release 4 is still in progress — the phase numbers are a map, not a schedule.
Renumbering notice. Releases 3+ were re-sequenced when the platform generalized to multi-schema contracts. The old numbering (R3 Schema Registry, R4 Observability, R5 Mutable Lifecycle, R6 Extensibility, R7 Operability) no longer applies; nothing was dropped, only re-ordered and, in two cases, folded into the registry work. The Release 1–2 sections below are the historical record of what was actually built and are unchanged.

How to read this

Phases are numbered R.P (Release.Phase) and listed in build order within a release.
Each phase names the repo(s) it touches, what to build, and a tiny, concrete done-when check — usually one test or one observable behavior, matching the project's test-first practice.
A phase that says "test-first" means: write the failing test in the same phase, before or alongside the implementation — not as a follow-up phase.
[ ] checkboxes are for tracking progress in this file as you go.

Spring Boot 4 / Jackson 3 gotchas hit so far

Discovered while building source-service (Phases 1.1–1.8), and worth watching for in db-adapter/file-adapter too (Spring Data JPA has had similar module-boundary changes in the same Spring Boot generation):

Gotcha	What actually happened	Fix
Web starter renamed	`spring-boot-starter-web` is now `spring-boot-starter-webmvc`	Use the new artifact id (Spring Initializr will get this right if you regenerate)
Test starters split per-feature	The old monolithic `spring-boot-starter-test` is gone; each feature has its own `-test` starter (`spring-boot-starter-webmvc-test`, `-kafka-test`, `-actuator-test`, `-validation-test`, `-restclient-test`, …)	Add the specific `-test` starter for whatever you're slice-testing; missing one shows up as a compile error or a missing bean, not a clear message
Test annotations moved packages	`@WebMvcTest` → `org.springframework.boot.webmvc.test.autoconfigure`; `TestRestTemplate` → `org.springframework.boot.resttestclient`	Search the actual jar (`unzip -l ... \| grep ClassName.class`) rather than guessing the package — guessed imports fail with "package does not exist," not a helpful redirect
`TestRestTemplate` no longer auto-registered	`@SpringBootTest(webEnvironment = RANDOM_PORT)` alone doesn't give you a `TestRestTemplate` bean anymore	Add `@AutoConfigureTestRestTemplate` and the `spring-boot-starter-restclient-test` dependency (it needs `RestTemplateBuilder`, which nothing else pulls in)
Spring Boot's own default Jackson is now Jackson 3	`tools.jackson.core`, not `com.fasterxml.jackson.core` — but Spring Kafka's `JsonSerializer` still uses legacy Jackson 2.x internally	Add `com.fasterxml.jackson.datatype:jackson-datatype-jsr310` (the 2.x one) explicitly, or `java.time` fields fail to serialize with an `InvalidDefinitionException` about missing modules
Jackson still defaults to `WRITE_DATES_AS_TIMESTAMPS`	Without disabling it, `java.time` fields serialize as numeric arrays/epoch numbers, not the ISO-8601 strings this project's canonical examples show	Build the Kafka value-serializer's `ObjectMapper` with `JavaTimeModule` registered and `WRITE_DATES_AS_TIMESTAMPS` disabled — via `DefaultKafkaProducerFactoryCustomizer` (`org.springframework.boot.kafka.autoconfigure`), not a hand-built `ProducerFactory`, so Spring Boot's own connection-detail handling stays intact
`@ServiceConnection` bypasses `spring.kafka.bootstrap-servers`	Spring Boot wires the Testcontainers address directly into the autoconfiguration; the property itself still just reflects `application.yml`'s fallback	Any raw test Kafka client needs its address from the `KafkaContainer` bean directly (`kafkaContainer.getBootstrapServers()`), never from `@Value` on that property
Initializr's `bootVersion` id has a `.RELEASE` suffix that isn't a real artifact version	`4.1.0.RELEASE` doesn't exist on Maven Central; the actual version is `4.1.0`	Strip the suffix from the generated `pom.xml`'s parent version before the first build
`JsonDeserializer` trusts the producer's `__TypeId__` header by default	A cross-service consumer (`db-adapter`) can't resolve a header naming the producer's (`source-service`'s) Java class, which isn't even on its classpath	`.ignoreTypeHeaders()` and always deserialize to the consumer's own local DTO — also the architecturally correct choice, since the JSON contract crossing the boundary shouldn't imply a shared Java type
`@Modifying @Query` methods aren't auto-transactional	Unlike `save()` (transactional via `SimpleJpaRepository`), a custom native update/insert query throws `TransactionRequiredException` without one	Add `@Transactional` on the repository method itself

The common thread: every one of these was invisible until something actually ran — compiling cleanly, and even a mocked unit test passing, told us nothing. Real Testcontainers integration tests and live docker compose checks caught all of them.

Release 1 — Walking Skeleton

Goal: docker compose up, submit one intern through the real UI, watch it land in Postgres and the CSV, with basic reliability (idempotency, isolated failures, a working DLQ) already true — not bolted on later.

Source Service

1.1 — Canonical record type. source-service: add a CanonicalInternRecord (Java record) with the fields from Data Model §1 — no logic, just the shape. Done when: it compiles and a trivial construction test passes.
1.2 — Intake DTO + validation. source-service: CreateInternRequest DTO with Bean Validation annotations (@NotBlank, @Email, @NotNull on startDate, etc.). Done when: a validator unit test shows a request missing email fails validation and a fully-populated one passes.
1.3 — Canonical mapper. source-service: CanonicalMapper mapping CreateInternRequest → CanonicalInternRecord, generating recordId (UUID) and createdAt server-side. Done when: a unit test asserts two calls with identical input produce different recordIds and both createdAts are non-null.
1.4 — Kafka topics as code. source-service: NewTopic beans for intern.created (3 partitions) and intern.dlq (1 partition), matching Architecture §10 — makes topic creation part of the app, not just infra's manual init job. Done when: app starts against a real broker and the topics exist with the right partition counts (verify via kafka-topics.sh --describe, same check already used against infra).
1.5 — Kafka publisher. source-service: KafkaEventPublisher wrapping KafkaTemplate<String, CanonicalInternRecord> (JSON), publishing keyed by internId. Done when: a unit test with a mocked KafkaTemplate verifies the key and topic passed to send(...).
1.6 — POST /interns. source-service: InternController wiring @Valid request → mapper → publisher, returning 202 Accepted with recordId; 400 with field errors on invalid input. Done when: a @WebMvcTest/slice test covers both the happy path and a validation-failure path.
1.7 — Publish integration test. source-service: Testcontainers-Kafka test hitting POST /interns (real HTTP call, real embedded broker) and asserting exactly one message on intern.created with the correct key. Done when: that test passes and would fail if the key were wrong (mutate it locally to confirm the test actually catches it, then revert).
1.8 — GET /interns read path. source-service: simplest correct read model owned by the Source Service itself (e.g. an in-memory or JPA-backed store populated at publish time) — not a query against Postgres/the CSV, preserving the loose-coupling boundary (Architecture §4.2). Done when: submitting via POST and then calling GET /interns shows the new record.

Reviewed 2026-07-22 (Phases 1.1–1.8): all eight done-when checks re-verified — 23/23 tests green, the actual Docker image (not just mvn spring-boot:run) rebuilt and exercised against a fresh Kafka via docker compose, POST/GET round-tripped correctly, and the message read back off the real topic. One genuine Definition of Done gap found: item 5 (recordId/traceId in logs at every new step) isn't met yet — there's no application-level logging at all in source-service so far. Not blocking (full traceId propagation is explicitly Release 4 / Phase 4.7 work), but plain recordId logging could reasonably start now rather than waiting; left as an open gap rather than silently fixed, since it wasn't part of any phase's stated done-when criteria. Two doc inaccuracies fixed as part of this review: Architecture §5's Source Service diagram called the read model InternQueryRepository (built as InternRecordStore) and didn't show ApiExceptionHandler; UC-1 step 5 stated Schema Registry validation as already true (it's Release 3+, not yet built).

UI

1.9 — Intern form. ui: replace the walking-skeleton placeholder with a real form (fields matching the canonical model) with client-side validation, calling POST /interns via VITE_API_BASE_URL. Done when: submitting a valid form in the browser gets a 202 and shows a confirmation; an invalid one shows inline errors without calling the API.
1.10 — Records view. ui: a list view calling GET /interns, rendering submitted records. Done when: a freshly submitted record appears in the list after a refresh.

Verified 2026-07-22 (Phases 1.9–1.10): both done-when checks confirmed against the real running stack (Kafka via infra docker-compose, source-service on localhost:8080, ui dev server) with a Playwright browser session, not just a build check. An empty-form submit produced 7 inline client-side errors and no API call; a valid submit returned 202 with a confirmation banner and the record appeared in the list without a manual refresh; light/dark themes (Catppuccin Latte/Mocha) and layout responsiveness at desktop (1400px), tablet (820px), and mobile (390px) widths were all checked, including the dark-mode preference persisting across a reload. One real bug found and fixed along the way: Spring Boot doesn't enable CORS by default, so the browser silently discarded the API's response even though the server processed it (confirmed via curl — 202 with no Access-Control-Allow-Origin header); fixed with a WebMvcConfigurer (WebConfig) in source-service scoped to /interns/**, with allowed origins externalized via iip.cors.allowed-origins. All 23 source-service tests re-run green after the fix.

Database Adapter

1.11 — JPA entity. db-adapter: Intern entity mapped to the existing interns table (schema in the infra repo's postgres/init.sql) + a Spring Data repository. Done when: the app starts against Postgres with ddl-auto: validate passing (no mismatch).
1.12 — Kafka consumer (naive). db-adapter: @KafkaListener on intern.created, JSON-deserialize, map to entity, save(). Done when: publishing one message via the CLI or a test results in one row.
1.13 — Make it idempotent. db-adapter: replace save() with a native INSERT ... ON CONFLICT (record_id) DO NOTHING (@Modifying @Query). Test-first: write the "deliver the same message twice, assert exactly one row" Testcontainers test before making it pass. Done when: that test passes.

Verified 2026-07-22 (Phases 1.11–1.13): all three done-when checks confirmed via real Testcontainers Postgres + Kafka, not mocks. 1.11: init.sql (mirrored from infra since this repo can't reference a sibling repo's file) applied to a real Postgres, ddl-auto: validate passed, entity save/find round-tripped correctly including a null mentor. 1.12: a raw JSON message published directly to intern.created (not via any shared Java type -- db-adapter must never depend on source-service's classes, only the canonical JSON contract) was consumed, deserialized, and landed as one row. 1.13, genuinely test-first: first confirmed what the naive save() from 1.12 actually did under duplicate delivery -- it already avoided a second row, but only incidentally (a non-null @Id makes Spring Data treat the entity as an update, not an insert, which isn't the same as a guaranteed atomic constraint) -- then replaced it with the explicit native ON CONFLICT (record_id) DO NOTHING upsert the phase calls for, confirmed via a real double-publish test. One gotcha hit and fixed along the way: @Modifying @Query methods aren't automatically transactional the way save() is via SimpleJpaRepository -- needed an explicit @Transactional on the repository method, or every call failed with TransactionRequiredException.

File Adapter

1.14 — Dedup store + CSV writer. file-adapter: a local dedup store (recording processed recordIds — an embedded file/SQLite is fine for R1) and a CsvInternWriter appending lines matching Data Model §4.2. Done when: a unit test writes two different records and asserts two distinct CSV lines.
1.15 — Kafka consumer + idempotent append. file-adapter: @KafkaListener on intern.created: check the dedup store, skip-or-append-then-record. Test-first: "deliver the same message twice, assert exactly one CSV line." Done when: that test passes.

Reviewed 2026-07-22 (Phases 1.1–1.15, full re-check): every phase's done-when re-verified, not just re-read. Full suites re-run clean: source-service 23/23, db-adapter 5/5, file-adapter 10/10; ui build + lint clean. Then a genuine end-to-end pass against the real Docker images (docker compose up --build, not mvn spring-boot:run) — submitted one intern both via raw HTTP and via a real browser session against the UI, and confirmed the identical recordId landed correctly in all three places: GET /interns (Source Service's own read path), a Postgres row, and a CSV line, with matching data in each. Two real doc inaccuracies found and fixed during this pass: Architecture's tech stack table claimed the file adapter's dedup index was "RocksDB/SQLite," but what's actually implemented (deliberately) is a flat file — file adapter is single-writer (AD-6), so there's no concurrent-writer race an embedded database would be needed for; and UC-5 claimed Schema Registry validation in its main flow, the same Release-1-vs-Release-3+ gap already caught and fixed for UC-1 in the earlier 1.1–1.8 review, just missed here since db-adapter didn't exist yet at that time.

Verified 2026-07-22 (Phases 1.14–1.15): all done-when checks confirmed. 1.14: plain unit tests (no Testcontainers needed) covered the literal done-when plus two edge cases worth locking in — a college name containing a comma is RFC4180-quoted rather than corrupting the column count, and reopening an existing CSV file doesn't duplicate the header. DedupStore is a flat file (one recordId per line), not an embedded database — file-adapter runs as exactly one instance (Architecture AD-6), so there's no concurrent-writer race a heavier store would be needed for. 1.15: a real Testcontainers Kafka integration test published one message (one CSV line) and then the same message twice (still one line). One real bug caught during this phase, before it reached a commit: the bare context-load test (FileAdapterApplicationTests) had no path override, so CsvInternWriter/DedupStore beans fell back to application.yml's default ./data/... paths and wrote a real file into the repo's working directory on every test run — caught when data/interns.csv showed up as an untracked file about to be committed. Fixed by isolating that test with a @TempDir (matching the pattern already used in the Kafka consumer test) and adding data/ to .gitignore.

Reliability (still Release 1 — see Implementation Plan §4, not deferred)

1.16 — Failure isolation test. db-adapter + file-adapter: Testcontainers test that stops Postgres mid-stream and asserts the File Adapter still succeeds independently while the DB Adapter's offset stays uncommitted (message not lost), then recovers once Postgres comes back. Done when: that test passes.
1.17 — Shared FailureClassifier. db-adapter + file-adapter: a small classifier distinguishing retriable vs. non-retriable exceptions, used identically in both adapters (Architecture §6). Done when: a table-driven unit test covers at least "connection refused → retriable" and "deserialization error → non-retriable."
1.18 — Bounded retry. db-adapter + file-adapter: wrap the write in the classifier + a bounded retry with backoff (config via application.yml, not a hardcoded constant). Done when: a test asserts the retry count and that it gives up after the configured max.
1.19 — DLQ on exhaustion/non-retriable. db-adapter + file-adapter: on non-retriable failure or retry exhaustion, publish the original message + error metadata (Data Model §3) to intern.dlq, then commit the offset. Test-first: "publish a malformed message, assert it lands in intern.dlq and a subsequent good message still processes." Done when: that test passes for both adapters.

Verified 2026-07-22 (Phases 1.16–1.19): implemented in dependency order (1.17 → 1.18 → 1.19 → 1.16) since each builds on the last, then committed in the phase numbering order shown above. db-adapter: 18/18 tests green (was 10). file-adapter: 19/19 tests green (was 10). Both adapters now deserialize the Kafka message by hand (ObjectMapper.readValue inside the listener) instead of via a Kafka JsonDeserializer — this was a deliberate design change from 1.12/1.15, not just an addition: it's what lets a malformed payload be classified and routed to the DLQ by the same code path as a write failure, and it removed KafkaConsumerConfig's wildcard-capture workaround entirely in both adapters. FailureClassifier is genuinely two different implementations sharing one shape, not one shared library — db-adapter's retriable set is ConnectException/TransientDataAccessException, file-adapter's is ConnectException/generic IOException (the realistic transient-failure analogue for a local-disk target), each adapter-appropriate. One real bug caught by the classifier's own test: file-adapter's first draft checked the retriable branch before the non-retriable branch, and since Jackson's JsonProcessingException is itself an IOException subtype, the broad IOException retriable check silently shadowed the deserialization-error case — the deserialization-error test failed, not the connection-refused one, which is what caught it. 1.16 needed two attempts: stopping and restarting the same Testcontainers Postgres instance left connections failing for 60+ seconds after start() returned (container restart isn't a reliable simulate-recovery technique in practice); switched to Docker pause/unpause, which freezes the container without tearing down its network binding, and that worked cleanly. file-adapter doesn't have its own 1.16 test — "the other target is unaffected" is a structural guarantee of Kafka consumer groups (separate groups on the same topic never block each other), already evidenced by file-adapter's suite never touching Postgres at all; inventing a test that touches Postgres from file-adapter's repo would have been testing Kafka's architecture, not this codebase.

Proof & demo

1.20 — Full end-to-end test. One Testcontainers test: publish once (via the real HTTP path), assert a row appears in Postgres and a line appears in the CSV — the proof test called out in Original Specification §9.

Verified 2026-07-22 (Phase 1.20): lives in infra/e2e-tests, a standalone Maven+JUnit+Testcontainers project (no Spring Boot — it only orchestrates other services' containers), since the test spans three separate Maven projects that no single repo can @Import together. Builds the real Docker images for source-service, db-adapter, and file-adapter from their own Dockerfiles via Testcontainers' ImageFromDockerfile — a stronger proof than an in-JVM multi-context trick, since it exercises the actual deployment artifacts, not just the source. One real bug hit and fixed: Testcontainers' KafkaContainer wrapper only advertises the host-mapped address, so every container-to-container connection (all three app containers, on this test's own Docker network) bootstrapped successfully against kafka:9092 and then failed on the very next coordinator-discovery step — the exact "Kafka advertised-listener trap" already documented in infra/README.md, just not yet hit from inside a container before. Fixed by dropping the KafkaContainer wrapper for a plain GenericContainer configured like docker-compose.yml's own proven-working listener setup, minus the host-facing listener this test never needs.

1.21 — Manual demo pass. infra: docker compose up --build, submit a real intern through the browser UI, confirm it in Postgres (psql) and the CSV file. Update infra/README.md with this as a documented smoke-test recipe.

Verified 2026-07-22 (Phase 1.21, expanded scope): the demo pass itself is documented in infra/README.md, but the phase grew into a real feature: a Targets page in the UI (ui's new nav tab) giving an operator view of both Release 1 targets, talking directly to each adapter's own new admin API (GET /interns, POST /admin/pause, POST /admin/resume, GET /admin/status — added to both db-adapter and file-adapter). Pause/resume controls each adapter's own Kafka listener via KafkaListenerEndpointRegistry, not the target container itself — a container can't pause itself (that would freeze the very HTTP thread handling the pause request), and pausing the listener proves the identical zero-data-loss guarantee: while paused, the consumer group's offset doesn't advance, so nothing published during the pause is lost, and the backlog processes once resumed. Clicking a target shows its actual data: a searchable table for Postgres, a spreadsheet-style grid (cell borders, row numbers, header shading) for interns.csv, deliberately styled to read as a distinct "raw file" view rather than another database table.
One severe bug found via a real docker-compose run (not either adapter's own test suite): both adapters' @KafkaListener used the identical id string for the new pause/resume lookup, and Spring Kafka's id doubles as the Kafka consumer group id by default unless groupId is set separately — so db-adapter and file-adapter silently joined the same Kafka consumer group, and Kafka split intern.created's partitions between them instead of each independently seeing every message, breaking the platform's core fan-out guarantee. Invisible to either adapter's own tests, since each spins up an isolated Kafka broker with only that one adapter ever connected. Caught because the Targets page's own zero-data-loss check failed in a way that didn't add up (a submitted intern never appeared in the Database view at all, not even after a long poll, while the same intern did appear in the File view) — fixed by setting groupId explicitly to each adapter's already-correct application.yml property. Full pipeline re-verified end-to-end in a real browser afterward: submit, pause Database, submit again, confirm absent while paused, resume, confirm it lands.

Release 2 — Reliability Hardening

2.1 — Formalize the classifier. Extract FailureClassifier (from 1.17) into a shared, explicitly-documented exception → classification table; expand the unit test table with more exception types as they're discovered in practice.
2.2 — Chaos test: Postgres. Formalize 1.16 into a repeatable chaos suite: kill/restart Postgres via Testcontainers mid-consumption, assert eventual consistency with no loss/duplication.
2.3 — Chaos test: Kafka reachability. Same for a Kafka broker disruption affecting the File Adapter.
2.4 — DLQ envelope completeness check. Audit the DLQ payload against the full schema in Data Model §3 (attemptCount, failedAdapter, timestamps, etc.) and fill any gaps left from 1.19.
2.5 — Retry/backoff externalized per-adapter. Confirm max attempts and backoff are independently tunable per adapter via config, not shared hardcoded values.

Verified 2026-07-24 (Phases 2.1–2.5): every FailureClassifier (both adapters) was rewritten from an if/else chain into an explicit List<ClassificationRule> evaluated first-match-wins across the cause chain -- the ordering-sensitivity bug Phase 1.17 found (a narrower non-retriable type being a subtype of a broader retriable one) is now a property of the table's declared order, not a comment asking the next editor to remember it. db-adapter's test table grew from 6 to 12 cases, file-adapter's from 4 to 7, including a genuine cause-chain-walk test file-adapter was missing (parity gap with db-adapter, unrelated to this release but fixed in passing).
2.2/2.3 each got a new two-test chaos suite (PostgresChaosTest in db-adapter, KafkaReachabilityChaosTest in file-adapter) covering both properties 1.16 only asserted for a single message: eventual consistency with no loss (every message published during the outage still lands) and no duplication (idempotency holds even when retries and redelivery are both in play at once, including a "flapping" variant that disrupts the target more than once during a single message's retry window). Both use Docker pause/unpause, not literal kill/restart, deliberately preserving 1.16's already-hard-won finding that stop/restart of a single Testcontainers instance is unreliable -- the phase's own "kill/restart" wording is satisfied in spirit (a genuine, repeatable disruption-and-recovery cycle), not lifted literally over a technique already proven flaky.
2.4's audit was the most consequential part of this release -- it started as "check the fields are all present" (they were: both DlqEnvelopes already matched Data Model §3 exactly) and surfaced four real, previously-invisible bugs, none caught by any existing test because Phase 1.19's DlqTest only ever exercised one path (a malformed message, immediately non-retriable, never retried):
DlqPublisher.errorType() labeled any unrecognized exception "RETRY_EXHAUSTED" regardless of whether it had actually been retried -- an immediately non-retriable failure on attempt 1 was indistinguishable from a target that failed for the entire retry budget, defeating the envelope's own stated purpose ("the failure is fully diagnosable"). Fixed by asking the classifier first: RETRIABLE → RETRY_EXHAUSTED, NON_RETRIABLE → a specific label or the new UNCLASSIFIED_FAILURE fallback (documented in Data Model §3).
Both adapters' JacksonConfig ObjectMapper never disabled WRITE_DATES_AS_TIMESTAMPS -- the exact gotcha already in this document's own table, just never applied to this ObjectMapper (only the Kafka producer's). quarantinedAt was serializing as a raw epoch-second double (1.784874960562927E9), not the ISO-8601 string the data model specifies. Caught only because a new test finally parsed it as an Instant instead of treating the envelope as an opaque string.
Proving a genuine RETRY_EXHAUSTED path required a Postgres outage long enough to exhaust every attempt (not a brief 1.16-style blip), and that surfaced three distinct exception types FailureClassifier didn't recognize -- all real, all from the @Transactional machinery rather than org.springframework.dao, none exercised by any prior test: CannotCreateTransactionException (transaction couldn't be opened -- no connection available), TransactionSystemException (the write failed and the subsequent rollback also failed against the same broken connection -- Spring logs "Application exception overridden by rollback exception" and this is what actually propagates), and Hibernate's own TransactionException (the same failure surfacing wrapped in Spring's generic JpaSystemException instead of being translated to either rule above). All three are now explicit, documented RETRIABLE rules with the reasoning for why each is safe to retry (none of them are about the data).
A genuine production reliability gap, not just a test artifact: db-adapter had no PostgreSQL driver-level connectTimeout/socketTimeout configured at all (both default to 0 / no timeout). Against a target that's genuinely unresponsive rather than actively refusing (a paused container's kernel-level TCP stack can still ACK the initial SYN, since the accept queue is kernel-managed independent of the frozen process -- the hang is waiting for a protocol-level response only the frozen process could send), a single connection attempt was measured at 25-28 seconds, dominated by the OS's own tcp_syn_retries behavior rather than anything Hikari's connection-timeout bounds. Fixed with explicit spring.datasource.hikari.data-source-properties.connectTimeout/socketTimeout defaults in application.yml (both externalized via env vars), which is a real hardening improvement independent of this release's test suite -- a truly unresponsive Postgres in production should fail fast into the retry/DLQ path, not hang each attempt for however long the kernel's default happens to be.
DlqEnvelopeTest (new, both adapters) proves the full envelope by parsing real JSON off the real DLQ topic (not substring checks like 1.19's test) and specifically forces the RETRY_EXHAUSTED path: db-adapter's version pauses Postgres for a fixed window sized to the ~27s-per-attempt reality above; file-adapter's points iip.file.output-path at an existing directory, so Files.writeString fails with a persistent, always-retriable-per-classifier FileSystemException that never recovers on its own -- a deliberately different, simpler failure-forcing technique than Docker pause, since file I/O failures don't have the same kernel-level TCP wrinkle Postgres does.
2.5 turned out to already be true at the application level (each adapter's RetryProperties is @ConfigurationProperties(prefix = "iip.retry") in its own Spring context -- structurally incapable of sharing state even if someone wanted to) but was silently false at the deployment layer: infra/docker-compose.yml never passed RETRY_MAX_ATTEMPTS/RETRY_INITIAL_BACKOFF_MS/RETRY_MULTIPLIER into either adapter's environment at all, so no .env override could ever reach them (they'd have silently used application.yml's bare defaults regardless of what .env said). Fixed with distinct DB_ADAPTER_RETRY_*/FILE_ADAPTER_RETRY_* variable names (not one shared RETRY_* name, which couldn't hold two different values for two services at once) -- the same class of gap independently found and fixed for CORS_ALLOWED_ORIGINS earlier in this engagement, this time caught proactively via this phase's own "confirm" wording rather than via a live bug report.

Release 3 — Contract Extraction + Envelope Split

Goal: make the existing intern pod a config-driven instance rather than a hardcoded one — the cheapest possible proof of parameterization, and the step that commits to neither Path A nor Path B (06 §4). Nothing user-visible changes in this release; that's the point.

3.1 — Envelope type. source-service: add a CanonicalEnvelope type with the fixed fields from Data Model §1a (recordId, contractId, recordType, schemaVersion, naturalKey, occurredAt, traceId, payload) and move the existing intern fields down into its payload. Two of 1.1's fields go up rather than down — recordId and createdAt (as occurredAt) were always envelope-level concerns and become envelope fields, leaving the payload as exactly the nine business fields the contract declares (Data Model §1b). Nothing is renamed, retyped, or lost. Done when: a unit test asserts the envelope round-trips through JSON, and that re-flattening it (envelope recordId + payload fields + occurredAt as createdAt) reproduces 1.1's wire shape byte for byte.
3.2 — Adapters read the envelope. db-adapter + file-adapter: deserialize the envelope and pull the intern fields from payload rather than the top level, taking recordId/occurredAt from the envelope. Done when: both adapters' full suites pass unchanged in assertions (only the fixture shape moves) — a real end-to-end submit still lands one Postgres row and one CSV line, and the CSV's columns are byte-for-byte what they were.

Verified 2026-07-27 (Phases 3.1–3.2): all three suites green against real Testcontainers infrastructure — source-service 31/31 (was 23), db-adapter 20/20, file-adapter 24/24 — with no existing assertion weakened or deleted to accommodate the new shape. The five tests added are all in the new CanonicalEnvelopeTest; the three added to CanonicalMapperTest cover envelope fields that didn't exist before (contract identity, key derivation, unset traceId).
A doc conflict surfaced and was resolved in favour of the data model. 3.1's original done-when asked for the payload's JSON to be "byte-identical to what 1.1's record produced," which cannot hold alongside Data Model §1b: the interns payload is exactly nine business fields, and recordId/createdAt are envelope-level. Rather than duplicate those two fields on both levels — two sources of truth, guaranteed to drift — the payload follows §1b and the done-when was rewritten to the check that actually proves nothing was lost: reflatteningReproducesTheReleaseOneWireShapeExactly reconstructs the flat Release 1 JSON from recordId + payload + occurredAt and asserts it byte for byte. That test is the one that would catch a renamed or dropped field; the original wording would not have, since it never compared against the old shape at all.
Two decisions worth recording. InternsContract holds the three hardcoded values (contractId, recordType, schemaVersion) in one named, explicitly-temporary class rather than as literals scattered through CanonicalMapper — Phase 3.3 deletes it wholesale instead of hunting string literals. EnvelopeJsonFixture (one per adapter, test scope) exists because four separate adapter tests had each inlined their own copy of the flat wire shape; without it, 3.2 would have meant editing the same JSON literal four times per adapter, and Releases 5–7 would each mean editing it four more. The fixture moved; every assertion around it stayed.
The GET /interns HTTP response shape is deliberately unchanged — InternSummaryResponse still flattens recordId + payload + createdAt — so ui needed no changes and the wire-contract split stays invisible to clients. infra/e2e-tests likewise needed no changes: it drives the real POST /interns intake shape (untouched) and asserts on Postgres rows and CSV lines (untouched), which is precisely the property 3.2 claims.
One process note, since it cost a full re-run: running db-adapter and file-adapter suites concurrently made a Kafka Testcontainer exit 1 with "Timed out waiting for log output matching .*Transitioning from RECOVERY to RUNNING.*" — a resource-contention failure with no test code involved, which is easy to misread as a real regression. Run the adapter suites serially.

3.3 — Contract file. source-service: add the interns contract file from Data Model §1c to the image and a ContractLoader that reads it at boot. Done when: the app fails fast at startup with a clear error if the file is missing or malformed, and logs the loaded contractId + schemaVersion on success.
3.4 — Contract-driven validation. source-service: PayloadValidator validates the submitted payload against the loaded contract — field presence, types, enum domain. Test-first. Done when: removing a required field from the contract file makes a previously-valid submission return 400, with no recompilation — this is the phase that actually proves parameterization, so don't let it pass on a mock.
3.5 — Contract-driven key + mapping; delete the compiled schema. source-service: derive naturalKey via the contract's key strategy, then delete CanonicalInternRecord, CanonicalMapper, and CreateInternRequest's Bean Validation annotations. Done when: the compiled intern schema no longer exists in the codebase — not merely bypassed (Implementation Plan §7 risk register: a fallback that still compiles is a fallback that will be used).
3.6 — Topic rename to the derived scheme. All services + infra: intern.created → interns.created, intern.dlq → iip.dlq, topic names derived from contractId rather than hardcoded (Architecture §10). Rename only — the partition key's runtime value is unchanged, since naturalKey for interns is internId. Done when: docker compose up on a clean volume produces the new topic names and the full e2e test passes against them.
3.7 — Regression gate: the intern pod behaves identically. Re-run every Release 1–2 suite plus infra/e2e-tests against real Docker images, and a manual browser submit. Done when: all green with no test weakened to accommodate the change. ← parameterization proven.
3.8 — Forms as contract #2. Add a forms contract file (no Java changes at all) and submit a forms record. Done when: it validates, publishes to forms.created, and the only diff in the repo is one new file. If any Java change was needed, 3.4/3.5 aren't actually finished — go back rather than patching forward.

Verified 2026-07-27 (Phases 3.3–3.8): every automated gate green — source-service 48/48 (was 31), db-adapter 32/32, file-adapter 32/32, and infra/e2e-tests 1/1 against real Docker images rebuilt from this source. No test was weakened or deleted to accommodate the change. Ten test methods were removed, but only because the classes they tested no longer exist (3.5 deleted CanonicalInternRecord, CanonicalMapper, CreateInternRequest, InternRecordStore); every behaviour they asserted is now asserted against the contract-driven path instead, and the net test count went up in all three services.
Two gaps found on a second pass, after the phases were first called done. Both were cases where a done-when was asserted rather than tested, which is the failure mode this document exists to prevent, so they're recorded rather than quietly fixed:
3.3's fail-fast claim was untested. ContractLoaderTest had seven tests, but five of them constructed Contract/FieldDefinition/FieldType directly and never touched ContractLoader — they tested the model's invariants, which says nothing about whether a broken file ever reaches them. The loader's own parse path (nine throw sites) and its missing-file path were entirely unexercised, while the done-when specifically says "missing or malformed". Fixed by making the scan location injectable and adding six tests over real broken YAML fixtures, asserting each error names the file and the offending token.
3.8's "publishes to forms.created" rested on a mock. A Mockito captor asserting which topic name was passed to a mocked KafkaTemplate is a weaker claim than the phase makes. Fixed by adding aSecondContractsRecordTravelsTheWholePathToItsOwnTopic, which validates → builds → publishes a forms record and reads it back off a real broker, asserting the composite key F-IT-1|Q-IT-9 and both declared defaults survive the round trip.
What 3.8 actually cost. One file in src/main — contracts/forms.yaml. No FormsController, no FormPayload, no forms mapper, no forms topic declaration, no if (contractId.equals("forms")). (Tests were added, of course; the phase's "only one new file" is a claim about production code, and that is how it should be read.) The forms contract deliberately exercises what interns cannot — a composite natural key ([formId, questionId] → "F-1|Q-7"), an integer, and a boolean — because a second contract shaped like the first would still pass against code that secretly assumed a single string key. KafkaTopicsConfigTest asserts forms.created is provisioned as a real Kafka topic, which is the infrastructure half of the same proof.
3.4's done-when was taken literally. makingAFieldOptionalInTheContractChangesValidationWithNoRecompile loads two contract files differing only in one required: flag and asserts the same payload is rejected by one and accepted by the other — no mock, no stub validator. That test is the whole point of the release; if it ever starts passing for the wrong reason, parameterization is gone.
One extension beyond Data Model §1c, now folded back into it: default. Release 1's CanonicalMapper hardcoded "a new intern always starts ACTIVE", and the UI has never sent status. Deleting that mapper in 3.5 left a choice between making status optional (weakening the contract) and making every client send it (changing the intake shape). Neither is acceptable for a release whose whole claim is "behaves identically", so the field model grew a default: applied before the required check. §1c now documents it.
A second, smaller extension: unknown fields are rejected, not ignored. A payload key the contract doesn't declare is almost always a typo or a client running ahead of the schema, and silently dropping it would hide data loss behind a 202. It also keeps the generic landing table (Data Model §4.0) from storing the typo forever.
One intern-shaped class survives 3.5 on purpose: InternSummaryResponse, the GET /interns read model, still names the nine business fields. It is outside 3.5's stated scope (which names CanonicalInternRecord, CanonicalMapper, and CreateInternRequest's annotations) and deliberately so — keeping the read shape frozen is what lets ui go untouched through the whole release. It generalizes in Phase 4.5 alongside GET /contracts/{id}/records. Nothing on the write path knows what an intern is.
As built vs as designed, on topics. Architecture §10 derives topics from contractId; the code now does exactly that (TopicNames.created(contractId)), and application.yml no longer carries a topic name at all — only partition counts, which are deployment-shaped rather than schema-shaped. The DLQ went the other way on purpose: iip.dlq is platform-wide rather than per contract, because it is an operator surface and every DLQ envelope already carries the contractId to filter on.
3.7's manual sign-off, 2026-07-29. The clean-volume run (docker compose down -v && docker compose up -d --build) and the UI submit were both done by hand, and the stack afterwards shows what the release claims. Kafka lists exactly interns.created, forms.created, and iip.dlq, with no intern.created surviving anywhere. Two interns submitted through the UI (INT001, INT002) landed in Postgres and in interns.csv with byte-identical record_ids, which is the envelope's identity holding across two independently-written adapters. iip.dlq sat at offset 0 throughout — nothing dead-lettered. Both rows read status=ACTIVE although the UI has never sent status, so the default: that replaced CanonicalMapper works in the real stack and not just in the validator's tests. And forms.created exists on a broker in a deployment where nothing has ever submitted a form: KafkaAdmin provisioned it from forms.yaml alone, which is 3.8's claim confirmed by infrastructure rather than by a test.
Two environment snags worth recording, neither a code fault. The ui service builds from ../ui, a separate repository (Azaken1248/iip-ui) that has to be cloned as a sibling of infra before docker compose up can succeed — a fresh checkout of the backend repos alone fails the build with path "/home/aza/IIP/ui" not found, after the three Java images have already built. And the UI's default host port 3000 is a popular one; when it is taken, either free it or set UI_HOST_PORT and add the new origin to CORS_ALLOWED_ORIGINS, since source-service will otherwise serve the page and then reject every submit it makes.
One process note carried over from 3.1–3.2, now understood properly: the Kafka Testcontainer failure "Timed out waiting for log output matching .*Transitioning from RECOVERY to RUNNING.*" is memory pressure, not a code fault. It reappeared here and was fixed by capping the fork heap (MAVEN_OPTS=-Xmx512m) and running the adapter suites serially. It involves no test code at all and is easy to misread as a real regression.

Release 4 — Parameterized Source Service + Contract Registry

Goal: the contract stops being a file in an image and becomes data in a service, and the envelope becomes a registry-enforced contract. Absorbs the former "Contract Enforcement" release.

Decision recorded during 4.6–4.8 — the Schema Registry owns the schema, not the wire. The phases ask for a validating serializer and validating deserializers. Confluent's own serializers would supply that by framing every message with a magic byte and a schema id, which is the canonical implementation and was rejected here. Framing would make every adapter's bytes depend on the registry being reachable and on a Confluent client library, undoing the property held since Phase 1.12: an adapter reads canonical JSON off a topic and needs no shared artifact to understand it. What was built instead keeps plain JSON on the wire and puts the registry where the value is — infra/schemas/envelope.json is registered as subject iip.envelope-value under BACKWARD compatibility, all three services fetch it at startup and refuse to start without it, and every message is checked against it on the way out and on the way in. The governance process in Data Model §5 is unaffected: the compatibility check is still the registry's, and CI still runs it (infra/scripts/compatibility-gate.sh).
A real bug 4.11 caught, which is what the phase is for. Both adapters rejected the added optional field outright: Jackson enables FAIL_ON_UNKNOWN_PROPERTIES by default, so the hand-built ObjectMapper in each adapter's JacksonConfig refused any payload key its record did not declare. The whole control plane would have said yes — registry, compatibility gate, source service — and the change would have surfaced as one DLQ entry per record from services nobody had redeployed. Fixed by disabling the feature in both adapters, with a regression test in each; written up as producer strict, consumer tolerant in Data Model §5, since the source service's opposite behaviour is correct for the source service.
Also worth knowing. additionalProperties is deliberately left open on the envelope schema — BACKWARD exists so an adapter keeps working when the source service ships an envelope change first, and a closed schema would break exactly that case on the consumer side, in production. Each service keeps a copy of envelope.json under src/test/resources so its own suite runs without a registry; those copies are fixtures, and the CI gate fails if one has drifted, because a drifted fixture makes a suite that passes against a schema production does not use.

Release 5 — Config-Driven DB Adapter + Generic Landing Table

Goal: an arbitrary payload reaches Postgres with no redeploy and no DDL grant held by any service.

5.1 — records table. infra: the hybrid landing table from Data Model §4.0, with record_id PK and UNIQUE (contract_id, natural_key).
5.2 — Attachment read path. db-adapter: read adapter_attachments for adapter_type = 'postgres' at boot + on a refresh interval; log the contracts it is attached to.
5.3 — Contract filter. Skip-and-commit any envelope whose contractId isn't attached. Test-first: publish an unattached contract's record, assert no row written and the next attached record still processes (the partition must keep moving — this is the same liveness property as the DLQ, applied to a non-failure).
5.4 — Generic write path. Insert envelope columns + payload JSONB with ON CONFLICT (record_id) DO NOTHING. Test-first: "deliver twice, assert one row" — for a contract with no Java type anywhere in the repo.
5.5 — Natural-key upsert. ON CONFLICT (contract_id, natural_key) DO UPDATE for update-style record types. (Lifecycle events themselves are Release 7; this phase just makes the write path ready and tests it by publishing an update-style envelope directly.)
5.6 — Queryable fields become indexes. Fields a contract marks queryable: true get expression indexes over payload, created when the attachment is registered. Done when: EXPLAIN shows the index used for a payload->>'status' filter on contract_id = 'interns'.
5.7 — Shaped-table mode. Support a per-attachment "shaped table" config and point the existing interns table at it. Done when: both modes run side by side — interns in shaped mode, forms in records — proving the generic table is a default rather than a mandate.
5.8 — Forms lands end-to-end. Release exit criterion. Submit a forms record through the real HTTP path against real Docker images; assert it lands in records, with zero code written for forms specifically.

Decision recorded during 5.3 — the adapter subscribes to a topic pattern, not a list. A contract registered through the control plane gets a topic ({contractId}.created) that no deployment descriptor mentions, so a listener bound to named topics would need reconfiguring to see it — which would make UC-14's "no redeploy" false in the one place it has to be true. The adapter consumes every create stream and skips the contracts it isn't attached to. This forced one non-obvious setting: a pattern subscription cannot auto-create a missing topic the way an explicit one can, so a consumer that starts before its topics exist is assigned zero partitions and stays that way until the next metadata refresh — five minutes, by default, of a newly attached contract appearing not to work. metadata.max.age.ms is 5s in both adapters, because that interval is the latency of "attach a contract and watch it flow".
Decision recorded during 5.5 — no guard against a stale occurredAt overwriting a newer one. The natural-key upsert replaces the row wholesale, so an out-of-order update would lose data. It cannot arrive out of order: every event for one entity is keyed by naturalKey and therefore shares a partition, which is precisely why Architecture §5 chose that key. A timestamp guard would not add ordering, only mask a broken key strategy by turning "records are arriving out of order" into "some updates silently do not apply". Left out deliberately, recorded here so it reads as a decision rather than an oversight.
The DDL question the release goal raises, and how 5.6 answers it. The goal above says "no DDL grant held by any service", and 5.6 creates indexes at runtime. The distinction being drawn is the one Data Model §4.0 drew when it rejected DDL-per-contract: a service that owns table shape owns schema drift, ALTER TABLE migrations, and the ability to destroy data by getting one wrong. CREATE INDEX on one existing table changes no data, loses nothing if it is wrong, and is undone by dropping it. The adapter issues that and nothing else — never CREATE, ALTER or DROP TABLE — and index creation is never fatal: a deployment that withholds DDL rights entirely is a legitimate choice, so a permission error logs the exact SQL for an operator and the adapter carries on writing records.
What 5.7 removed, which is the release in one line. CanonicalEnvelope, InternPayload, and InternRepository's upsert are gone. The write path used to bind every message to a Java record whose fields were one contract's payload; it now works from the parsed tree in both modes, and the shaped mapping that named nine intern columns is a map in adapter_attachments.config. The adapter's src/main no longer contains a type for any contract's payload — asserted directly by a test that greps it, since a passing write test proves forms records land, not that they land generically. What stayed intern-shaped is the read side (GET /interns, which the UI calls); that generalizes in 6.9.
Attachments are seeded with SQL for exactly one release. Contracts are POSTed through the public API by contract-registry-init, because that API exists. Attachments have no API until 6.6, so infra/seed/attachments.sql is applied by an attachment-init job. Deliberately not an initdb.d script: an attachment references a contract by foreign key and the contracts are registered over HTTP long after initdb finishes, so as an init script it would take the Postgres container down on a fresh volume — and init scripts don't run on an existing one, which is the trap Release 4 already fell into once.

Release 6 — Adapter Catalog + Attachments + Control-Plane UI

Goal: the user-facing goal is met — define a schema and wire its targets from a UI, no developer, no redeploy.

Decision recorded during 6.1 — adapters register themselves. The catalog could have been a list in the registry, and UC-9 would still have been satisfied to the letter, since it names the UI, the source service and the existing adapters but not the registry. It would also have made 6.5 a lie in spirit: adding a type would mean editing a second repository, and that entry would be the first thing to go stale. So each adapter ships a static descriptor next to the code that reads the config it describes, and PUTs it at startup. A type exists because its service is running.
What the descriptor turned out to be worth. It started as documentation for a dropdown and became enforcement: 6.6 validates an attachment against the config schema its adapter published, so a webhook with no url is refused at attach time with the type's own description of the field quoted back — instead of surfacing one record at a time in the DLQ, with the adapter that could have explained it nowhere near the request that caused it.
Decision recorded during 6.2 — the shape is duplicated, not shared. "Extract the generic adapter pattern" reads like a library, and the docs point elsewhere: Architecture §6 calls its pipeline the "acceptance checklist for any new adapter", and 6.5 builds one in its own repo. This platform has no artifact repository, and a published jar would turn "a new adapter type is a new service" into "a new service plus a versioned dependency on ours". So every adapter carries the pipeline, byte-identical modulo package name, and two gate checks keep the copies honest — one comparing the classes character by character, one comparing the acceptance suite that tests them. The second matters more: a weakened copy of the checklist is worse than a drifted pipeline, because it is the thing that would have caught the drifted pipeline.
Three bugs that only running it could find, all in code that compiled and passed. jsonb does not preserve an object's key order — it sorts keys by length then bytewise — so the csv adapter's column mapping came back rearranged and it wrote rows that did not match the header above them; the mapping is an ordered array now, and the fixtures are shaped like something a deployment can actually produce. iip.file.dedup-store-path changed meaning from a file to a directory, and an existing deployment had a file exactly where the new version wanted a directory: it started cleanly and then sent every record to the DLQ with "Not a directory", so it migrates itself and carries the old ids over. And isContainerPaused() is true only once every assigned partition is paused, so a listener paused before its topics existed — the ordinary state of a newly attached contract — reported itself running forever.
The window nobody would guess. Between attaching a target and the adapter's next refresh, records are skipped by the contract filter and their offsets committed; they are not delivered to the new target afterwards. That is correct skip-and-commit behaviour and a genuinely surprising consequence, so the attach form says so and points at replay. Which in turn surfaced that Kafka had no volume at all — docker compose down destroyed every topic, taking both replay and the DLQ with it.

Release 7 — Generalized Lifecycle

Goal: the former "Mutable Lifecycle" release, re-expressed so it works for every contract at once.

Release 8 — Observability + Operability

Goal: the former Observability and Operability releases, now multi-contract aware. contractId as a metric dimension is what keeps this one dashboard instead of one per schema.

Release 9 — Path A: Per-Contract Instances (optional, gated)

Do not start this release unless hard isolation has become a real, stated requirement (AD-12). It is broken down here only so the option stays cheap and its cost stays visible. The user-facing outcome is identical to Path B.

9.1 — IIPInstance CRD. Define the custom resource: contractId, adapter attachments, resource limits.
9.2 — Operator reconcile loop. Reconcile a CR into topics, a configured source-service pod, and adapter pods from the catalog images.
9.3 — Prove the images are unchanged. Run a contract under Path A using the same image digests Path B runs. Done when: the diff between the two deployments is config and namespace only. If a code change was needed, AD-9's parameterization bet did not actually pay off, and that finding is worth more than the release.

Keeping this document honest

When a phase's actual implementation reveals the next one needs re-splitting or reordering, edit this file — it's a working checklist, not a frozen spec.
Every phase should map to one commit (or a small handful) in the relevant repo. If a phase is producing multi-day, multi-file sprawl, it wasn't tiny enough — split it before continuing, per the project's small-releases practice.
Re-check Implementation Plan §6 Definition of Done at each phase, not just at the end of a release.
From Release 3 onward, one extra check per phase: would this work for a second contract? If the answer is no, the phase isn't done — the point of the generalization track is that "add a branch for the new schema" stops being an available move. See Implementation Plan §5.

Phased Rollout Plan ​

How to read this ​

Spring Boot 4 / Jackson 3 gotchas hit so far ​

Release 1 — Walking Skeleton ​

Source Service ​

UI ​

Database Adapter ​

File Adapter ​

Reliability (still Release 1 — see Implementation Plan §4, not deferred) ​

Proof & demo ​

Release 2 — Reliability Hardening ​

Release 3 — Contract Extraction + Envelope Split ​

Release 4 — Parameterized Source Service + Contract Registry ​

Release 5 — Config-Driven DB Adapter + Generic Landing Table ​

Release 6 — Adapter Catalog + Attachments + Control-Plane UI ​

Release 7 — Generalized Lifecycle ​

Release 8 — Observability + Operability ​

Release 9 — Path A: Per-Contract Instances (optional, gated) ​

Keeping this document honest ​