System Type: Enterprise Integration
Patterns, failure modes, and anti-patterns for connecting, modernizing, and orchestrating enterprise systems.
1. Integration Patterns
Enterprise integration is the art of making systems talk to each other when they were never designed to. Every pattern trades coupling for something — throughput, consistency, simplicity, or debuggability. The right choice depends on what you can afford to lose.
The Topology Spectrum
| Topology | Description | Coupling | Failure domain | Debuggability | When it works |
|---|---|---|---|---|---|
| Point-to-point | Direct connections between each pair of systems. N systems → N×(N-1)/2 connections. | Tightest — every system knows every other system's interface | Localized but unpredictable — one system's outage cascades differently depending on who depends on it | Easy when there are 3 systems, impossible at 15 | Small number of integrations (<5 systems), throwaway prototypes, or when two systems genuinely have a 1:1 relationship that won't grow |
| Hub-and-spoke (ESB) | Central bus mediates all communication. Systems only know the bus. | Moderate — systems decouple from each other but couple to the bus | Centralized — the bus is a SPOF for everything | Good — the bus logs everything (in theory) | When you need protocol translation, message transformation, and routing in one place. When governance requires a central integration team. When the number of systems is moderate (10–30) and the integration team has the capacity. |
| Event mesh / event backbone | Distributed event infrastructure (Kafka, Pulsar, etc.). Systems publish and subscribe to topics. No central broker logic. | Loosest — producers don't know consumers exist | Distributed — broker failure is partial, consumer failure is isolated | Harder — requires distributed tracing, topic monitoring, consumer lag tracking | When you have many (30+) systems, event-driven architectures, and teams that can own their own consumers. When you need to replay events for recovery or new consumers. |
Synchronous vs Asynchronous
Synchronous (request/reply). System A calls System B and waits for a response. The caller's latency includes the callee's latency. The caller's availability depends on the callee's availability.
When it's right:
- The caller genuinely cannot proceed without the response (validate payment before confirming order).
- Latency requirements are well understood and the callee can meet them.
- The dependency graph is shallow (A calls B, not A calls B calls C calls D).
When it creates coupling nightmares:
- Deep call chains. If A→B→C→D, A's p99 latency is the sum of B, C, and D's p99 latencies, and A's availability is the product of their availabilities.
- Retry storms. If B is slow, A retries. B is slow because C is slow. Now A and B are both retrying against C, making C slower. This is a distributed system death spiral.
- Temporal coupling. Both systems must be running simultaneously. Maintenance windows become a coordination problem across every system in the chain.
Asynchronous (fire-and-forget / event-driven). System A publishes a message or event and moves on. System B processes it whenever it can.
When it's right:
- The caller doesn't need an immediate response (order placed → fulfillment can happen later).
- You need to absorb load spikes (queue buffers the burst).
- You want temporal decoupling (System B can be down for maintenance without affecting System A).
- Multiple consumers need to react to the same event.
When it creates coupling nightmares:
- When you need the illusion of synchronous behavior built on async primitives. If the user is staring at a spinner waiting for 6 async hops to complete, you've built a slow synchronous system with worse debuggability.
- When ordering matters and you haven't designed for it. Messages arrive out of order. Events get replayed. Idempotency isn't optional — it's a survival requirement.
- When error handling is an afterthought. A failed async message goes to a dead letter queue. Who monitors it? Who retries it? Who tells the user their order silently failed 3 hours ago?
Messaging Patterns
Request/reply. Caller sends a request message with a correlation ID and reply-to address. Callee sends a response to the reply address with the same correlation ID. This is synchronous semantics over asynchronous transport. Use when you need the decoupling benefits of async but the conversation semantics of sync. Be aware that reply correlation adds complexity — lost replies, timeout management, and orphaned requests all need handling.
Fire-and-forget. Caller sends a message and does not expect a response. The simplest async pattern. Use for notifications, audit logging, analytics events, or any action where the caller doesn't care about the outcome. The risk: "fire-and-forget" often becomes "fire-and-pray" when the message is actually important but nobody built monitoring for delivery failures.
Publish/subscribe. Publisher emits events to a topic. Zero or more subscribers receive them independently. This is the foundational decoupling pattern in enterprise integration. Each subscriber can process events at its own pace, in its own way, without the publisher knowing or caring. The risk: invisible coupling. Adding a new subscriber is easy. Understanding the full impact of changing an event schema when you have 14 subscribers across 6 teams is hard.
2. Legacy Modernization
Every legacy system is load-bearing. It exists because it does something the business needs, and it has survived because replacing it is harder than maintaining it. Respect this. The codebase is ugly because it has 15 years of business rules encoded in it, not because the original developers were incompetent.
Strangler Fig Pattern
The single most important pattern in legacy modernization. Named after the strangler fig vine that grows around a tree, eventually replacing it.
How it works:
- Put a facade (proxy, gateway, load balancer) in front of the legacy system.
- For each new feature or migration target, build the new implementation behind the facade.
- Route traffic for that feature to the new implementation.
- Repeat until no traffic goes to the legacy system.
- Decommission the legacy system.
Why it works: You never have a big-bang cutover. Each migration step is small, reversible, and independently valuable. The legacy system continues running for everything you haven't migrated yet.
What goes wrong:
- The facade becomes permanent. You route 80% of traffic to the new system and 20% stays on legacy forever because those last features are the hardest to migrate and nobody wants to touch them. The "temporary" facade is now permanent infrastructure that you maintain for years.
- Data synchronization. If both old and new systems write to different data stores, you need to keep them in sync during the migration. Dual-write is fragile. Change data capture (CDC) is better but adds infrastructure. Every data sync mechanism is a source of bugs during migration.
- Feature development on both systems. Business doesn't stop during migration. New features get added to the legacy system because "the migration isn't ready for that module yet." Now you're maintaining two systems and the migration target keeps moving.
- Loss of organizational will. Strangler fig migrations take months to years. Leadership changes. Priorities shift. The migration stalls at 60% complete and you're running two systems indefinitely. The strangler fig only works if someone with authority defends the migration budget quarter after quarter.
Practical guidance: Start with the highest-traffic, lowest-complexity endpoints. These give you the fastest return on investment and build organizational confidence. Leave the gnarliest, most business-logic-heavy modules for last — by then you'll understand the domain better and have a proven migration playbook.
Anti-Corruption Layer (ACL)
What it is. A translation layer between your new code and a legacy system's data model. The ACL speaks the legacy system's language on one side and your clean domain model on the other.
Why it exists. Legacy systems have legacy models — field names that made sense in 1997, overloaded columns that mean different things depending on a status flag, business rules encoded as stored procedures. If your new code adopts the legacy model, you've imported 20 years of technical debt into your clean architecture. The ACL prevents contamination.
Where to place it:
- At the boundary of your new service, wrapping calls to the legacy system.
- In a dedicated adapter service if multiple new services need to talk to the same legacy system.
- NOT in the legacy system itself. You don't modify the legacy system. You insulate from it.
What it translates:
- Data model mapping (legacy
CUST_REC.ACCT_TYP_CD→ newCustomer.accountType). - Protocol translation (legacy SOAP/XML → new REST/JSON or gRPC/protobuf).
- Semantic translation (legacy uses
status = 7to mean "active with restrictions" → new model has explicitisActiveandrestrictions[]). - Error translation (legacy returns
RC=-412→ new model returns structured error with code, message, and recovery action).
The discipline: The ACL must be a team responsibility, not an afterthought. Someone owns the mapping. When the legacy system changes (and it will — even "frozen" legacy systems get emergency patches), the ACL must be updated. Put integration tests at this boundary. They'll save you at 2am.
Branch by Abstraction
What it is. Introduce an abstraction (interface, adapter, feature flag) in the existing codebase that lets you swap between old and new implementations without forking the codebase.
How it works:
- Identify the seam where old implementation meets the rest of the code.
- Insert an abstraction layer at that seam (interface, strategy pattern, feature flag).
- Old implementation now lives behind the abstraction.
- Build new implementation behind the same abstraction.
- Toggle between implementations using configuration or feature flags.
- When new implementation is proven, remove the abstraction and the old code.
When to use instead of strangler fig: When the thing you're replacing is a library, module, or internal component rather than an externally-facing service. Strangler fig works at the HTTP/network boundary. Branch by abstraction works at the code boundary.
Database Decomposition
The hardest part of legacy modernization is the database. Legacy systems almost always have a single shared database that everything reads from and writes to. Decomposing it is surgery on a beating heart.
Strategies, in order of increasing risk:
Read replica with new schema. Create a read replica, build a new schema on top of it with views or materialized views. New services read from the new schema. Writes still go to the legacy database. Low risk, but you're still coupled to the legacy schema and you can't write through the new model.
Change data capture (CDC). Use CDC (Debezium, AWS DMS, or database-native replication) to stream changes from the legacy database into a new data store. New services own their own data store and get updates via the CDC stream. Medium risk — CDC introduces latency (seconds to minutes) and you must handle the eventually-consistent window.
Dual-write with reconciliation. New service writes to both old and new databases. Reconciliation job detects and fixes drift. High risk — dual-write without distributed transactions means you WILL have inconsistencies. Only use this as a transitional strategy with tight monitoring and automated reconciliation.
Cut over by domain. Migrate one bounded context at a time. Move all reads and writes for that context to the new database. Update the legacy system to call the new service for that context's data instead of querying the shared database directly. Highest effort but cleanest result. This is the end state you're working toward.
The shared database trap: The legacy database is the single largest source of coupling in most enterprises. Every system that queries it directly is coupled to every other system that writes to it. Schema changes break unknown consumers. Performance problems in one system's queries affect every system on the same instance. Decomposing the shared database is the most important and most difficult step in any legacy modernization. Plan for it to take longer than everything else combined.
The Big Bang Rewrite
Almost never appropriate. The failure rate of big bang rewrites is staggering. Netscape. Borland. Countless internal projects that were cancelled 18 months in with nothing to show.
Why it fails:
- The old system has 10 years of implicit requirements encoded in bug fixes, edge cases, and undocumented behaviors. The rewrite team discovers these one at a time, in production.
- The business cannot wait 18 months for the rewrite. Feature development on the old system continues, and the rewrite target keeps moving.
- The team underestimates the scope by 3-10x because the old system looks simple from the outside. The complexity is in the interactions, not the code.
The rare cases where it's appropriate:
- The old system is so small that a rewrite is weeks, not months.
- The old system's technology is truly unsupportable (no one on the market knows the language, the vendor is bankrupt, the hardware is end-of-life).
- The domain is well-understood and the requirements can be fully specified up front (this is rarer than people think).
- You can run old and new in parallel for an extended comparison period before cutting over.
Even in these cases, prefer a strangler fig approach. The risk of big bang is not technical — it's organizational. Projects that take 18 months and show no incremental value get cancelled.
3. API Gateway Patterns
The API gateway is the front door of your integration architecture. Done right, it's an enabling layer that simplifies every team's work. Done wrong, it's a coupling bottleneck that every team hates and no one can change.
Gateway as Integration Fabric
The gateway isn't just a reverse proxy. In enterprise integration, it's where worlds collide — mobile clients hitting SOAP backends, partner APIs needing authentication you don't control, internal services speaking gRPC while external consumers expect REST.
Core gateway capabilities for enterprise integration:
| Capability | What it does | Why it matters for integration |
|---|---|---|
| Protocol translation | SOAP↔REST, REST↔gRPC, XML↔JSON | Legacy systems speak SOAP. New services speak gRPC. Clients expect REST/JSON. The gateway translates so no one has to. |
| Authentication aggregation | Validates tokens from multiple identity providers, normalizes claims | The legacy system uses SAML. The new system uses OAuth2. Partners use API keys. The gateway normalizes all of them into a single internal auth context. |
| Rate limiting | Per-client, per-tenant, per-endpoint throttling | Prevents a misbehaving integration partner from overwhelming a backend that has no rate limiting of its own (which is every legacy system). |
| Response composition | Aggregates responses from multiple backends into a single response | The client needs data from 3 backends. Instead of 3 round trips, the gateway composes. But this is where complexity lives — see caveats below. |
| Schema validation | Validates request/response payloads against schemas | Catches malformed requests before they hit backends that return cryptic errors. |
| Circuit breaking | Stops sending traffic to failing backends | The legacy system is down. Without circuit breaking, every request queues behind it, exhausting gateway connections. |
Protocol Translation
Protocol translation is the gateway's highest-value integration capability and its biggest maintenance burden.
SOAP to REST. The most common enterprise translation. SOAP services have WSDLs that define operations, types, and bindings. Map SOAP operations to REST resources and verbs. Map SOAP faults to HTTP status codes. Decide how to handle SOAP headers (WS-Security, WS-Addressing) — most can be dropped if the gateway handles auth, but some carry business semantics (correlation IDs, routing hints) that must be preserved.
The trap: auto-generating REST APIs from WSDLs produces APIs that are "REST" in name only — RPC-style operations mapped to POST endpoints with no resource modeling. If you're building an API that external consumers will use, invest in designing a real REST API and manually mapping it to the SOAP backend.
REST to gRPC. Use gRPC-JSON transcoding (built into Envoy, available as gRPC-gateway in Go). Define your gRPC service with HTTP annotations and let the transcoding layer handle conversion. This works well for new services that want gRPC internally but must expose REST externally. Watch for streaming — gRPC server streaming doesn't map cleanly to REST without SSE or WebSockets.
The ongoing cost: Protocol translation means maintaining two interface definitions. The SOAP WSDL changes, you update the gateway mapping. The gRPC proto changes, you regenerate the transcoding config. Every translation layer is a surface area for bugs. Automate the generation of translation configs from source schemas wherever possible.
Backend for Frontend (BFF)
What it is. A dedicated backend service for each frontend type (web, mobile, partner API). Each BFF aggregates, transforms, and optimizes responses for its specific consumer.
When to use in enterprise integration:
- Different consumers need radically different views of the same data. The mobile app needs a compressed summary. The partner API needs the full record in a specific XML format. The internal dashboard needs real-time aggregations.
- You want to decouple frontend release cycles from backend service changes. The BFF absorbs backend changes and presents a stable interface to its consumer.
- You have multiple teams building multiple frontends and you want each team to own its API contract.
When to avoid:
- You have one or two consumers with similar needs. A single gateway layer with minor per-consumer logic is simpler.
- The BFF teams don't have backend engineering skills. A BFF is a backend service — it needs monitoring, error handling, deployment pipelines, on-call.
The Gateway as a Coupling Point
The gateway is a SPOF by design. All traffic flows through it. This creates risks:
- Organizational coupling. If one team owns the gateway, every integration change requires that team's involvement. The gateway team becomes a bottleneck. Mitigate with self-service configuration: each team pushes their own routing rules, rate limits, and translations through a GitOps pipeline. The gateway team owns the platform, not every route definition.
- Performance coupling. The gateway adds latency to every request. If the gateway does heavy transformation (XML parsing, response composition from 3 backends), it can become the performance bottleneck. Keep translation lightweight. Move heavy composition to BFF services or dedicated aggregation services behind the gateway.
- Blast radius. A bad configuration change to the gateway takes down everything. Use canary deployments for gateway config changes. Test in staging. Have an instant rollback mechanism. Treat gateway changes with the same rigor as database schema changes.
- Logic creep. The gateway starts as routing + auth. Then someone adds a business rule. Then another. Now the gateway has business logic that should be in services, and it's deployed as a monolith that's terrifying to change. The rule: the gateway does transport-level concerns (routing, auth, rate limiting, protocol translation). Business logic goes in services. Enforce this boundary ruthlessly.
4. Data Integration
Data integration is where enterprise integration projects go to die. Systems agree on APIs relatively easily. Getting them to agree on what data means, who owns it, and how to keep it consistent is where the political and technical challenges converge.
Canonical Data Model
What it is. A shared vocabulary and data model that all systems agree to use for exchanging data. Instead of N×(N-1) point-to-point translations, each system translates between its internal model and the canonical model.
The appeal: Reduces translation complexity from O(N²) to O(N). Provides a common language for cross-team communication.
The reality: Building consensus on a canonical model is a political exercise disguised as a technical one. Every team believes their model is the right one. The canonical model committee produces either a lowest-common-denominator model that loses important domain nuance, or an everything-and-the-kitchen-sink model that no system fully implements.
When it works:
- The domain is well-understood and stable (financial transactions, healthcare records with HL7/FHIR).
- There's a strong central architecture team with organizational authority to enforce adoption.
- The canonical model is versioned and evolved deliberately, not frozen.
When it fails:
- When "canonical" means "the ERP vendor's data model." That's not canonical — that's vendor coupling dressed up as architecture.
- When the model is defined once and never updated. The business evolves. The model doesn't. Teams work around it with extension fields, out-of-band data, and side channels.
- When no one enforces compliance. A canonical model that 3 of 12 systems actually implement is worse than no canonical model — it's a false promise of interoperability.
Practical guidance: If you can't get organizational buy-in for a full canonical model (you probably can't), focus on canonical models for specific integration contexts — a shared order model for the order processing domain, a shared customer model for customer-facing systems. Domain-scoped canonical models are achievable. Enterprise-wide canonical models are a career hazard.
Event-Carried State Transfer
What it is. Instead of querying a source system for data, each consuming system maintains its own local copy of the data it needs, updated by events from the source. When System A updates a customer record, it publishes a CustomerUpdated event containing the full (or relevant subset of) customer data. Systems B, C, and D consume the event and update their local copies.
Why it matters for enterprise integration:
- Eliminates runtime coupling. System B can serve customer data even when System A is down, because it has its own copy.
- Eliminates query coupling. System B doesn't need to understand System A's query interface, pagination, or rate limits.
- Performance. Local reads are always faster than network calls.
The costs:
- Eventual consistency. System B's copy of the customer may be seconds or minutes behind System A's authoritative record. The business must be able to tolerate this staleness window.
- Storage duplication. Every consumer stores its own copy. For small datasets, this is trivial. For large datasets, it's significant infrastructure.
- Schema evolution. When the event schema changes, every consumer must be updated. This is the same problem as the canonical model — amplified by the number of consumers.
- Debugging. "Why does System B show the wrong address for this customer?" Because the event was published 3 minutes ago and System B hasn't processed it yet. Or because System B's event consumer crashed and there's a 10,000-event backlog. Or because the event schema changed and System B is silently dropping fields it doesn't recognize.
Change Data Capture (CDC)
What it is. Capture changes from a database's transaction log (WAL, binlog, redo log) and publish them as events. Unlike application-level events (which require the application to remember to publish), CDC captures every change regardless of which code path wrote it — application code, stored procedures, manual SQL, ETL jobs.
Key tools: Debezium (open source, Kafka-based), AWS DMS, Oracle GoldenGate, Striim.
When it's essential for enterprise integration:
- The source system cannot be modified to publish events (legacy system, vendor software, regulated system).
- Multiple code paths write to the same database and you can't guarantee they all publish events.
- You need to capture the full change history, including changes made by stored procedures, triggers, or batch jobs.
The costs:
- CDC operates at the database level, not the domain level. You get
row inserted in CUSTOMER tablenotcustomer signed up for premium plan. Consumers must infer domain semantics from database operations. This is an impedance mismatch that gets worse as the schema becomes more normalized. - CDC is sensitive to schema changes. Adding a column is usually fine. Renaming a column, changing a type, or splitting a table can break the CDC pipeline and every downstream consumer.
- Transaction log retention. The database must retain its transaction log long enough for CDC to read it. If the CDC consumer falls behind (outage, slow processing), the database may purge the log entries it hasn't read. This requires monitoring and alerting on CDC consumer lag.
The Shared Database Anti-Pattern (and How to Migrate Away)
The anti-pattern: Multiple applications read from and write to the same database, using the database as their integration mechanism. No APIs, no events — just shared tables.
Why it's so common: It's the easiest possible integration. No API to design, no messaging infrastructure to run, no serialization to worry about. SQL is a universal query language. The database handles transactions and consistency. It works brilliantly until it doesn't.
Why it's toxic at scale:
- Schema coupling. Every application is coupled to the physical schema. Renaming a column requires coordinated changes across every application — many of which are owned by different teams, on different release cycles, and some of which are vendor software you can't change at all.
- Performance coupling. Application A's slow report query locks tables that Application B needs for real-time transaction processing. A missing index in Application A's code path causes timeouts in Application B.
- No encapsulation. Every application can read and write any data. There are no domain boundaries, no validation at the integration layer, no access control beyond database grants (which are typically too coarse). Business rules are duplicated across applications — and they diverge.
- Change paralysis. Nobody wants to touch the shared schema because nobody knows the full impact. The DBA becomes the bottleneck for every change. "We'll just add a new table" becomes the default, and the schema accumulates cruft for decades.
Migration path:
- Inventory. Catalog every application that reads from or writes to the shared database. Identify which tables each application uses and whether it reads, writes, or both. This is harder than it sounds — don't forget stored procedures, views, triggers, ETL jobs, and reporting tools.
- Identify domain boundaries. Cluster tables into bounded contexts. Customer tables, order tables, inventory tables — which application should own each cluster?
- Build APIs over owned data. For each bounded context, the owning application exposes APIs for the data it owns. Other applications migrate from direct database queries to API calls. This is the hardest step and takes the longest.
- Introduce CDC for read-heavy consumers. Applications that do heavy reporting or analytics on data they don't own should get a CDC feed into their own read-optimized store rather than querying the shared database.
- Restrict database access. As consumers migrate to APIs and CDC, revoke their direct database access. The database credentials are the coupling mechanism — removing them is how you enforce the boundary.
- Decompose the database. Once each bounded context is accessed only through its owning application, you can physically separate the data into distinct databases.
This migration takes years in large enterprises. That's normal. The key is to make progress monotonically — each step reduces coupling, and no step makes things worse.
Data Mesh Concepts
Domain-owned data products. Instead of a central data team owning all data, each domain team owns its data as a product — with an SLA, a schema contract, documentation, and discoverability. Integration consumers treat data products like they treat APIs: call the product, get the data, trust the contract.
What this means for enterprise integration:
- The "integration team" doesn't own the data or the transformations. Domain teams own the data, expose it as products, and are accountable for quality.
- Central infrastructure provides the platform (catalog, access control, monitoring), not the data itself.
- Integration becomes API-like: discover the product, understand the contract, build your consumer. The organizational model shifts from centralized ETL to federated data ownership.
The prerequisite: Domain teams must be mature enough to own their data products. This requires investment in data engineering skills across teams, not just in a central data team. Many organizations aren't ready for this, and forcing it creates data products that nobody maintains.
5. Message Translation and Routing
Enterprise integration means connecting systems that were never designed to understand each other. Message translation and routing is where you bridge the impedance mismatch between different domain models, protocols, and data formats.
Message Transformation
What it is. Converting a message from one format/schema to another. This is the daily bread of enterprise integration.
Levels of transformation:
- Structural. Changing the shape of the data — XML to JSON, flat record to nested object, one schema to another. Mechanical and automatable.
- Semantic. Changing the meaning — mapping
status = "A"tostatus = "active", converting currencies, translating between code systems (ICD-10 to SNOMED). Requires domain knowledge and is where bugs live. - Enrichment. Adding data from external sources — receiving an order with a customer ID, looking up the customer's address from a customer service, and attaching it to the message before forwarding. Introduces a runtime dependency on the enrichment source.
- Filtering. Removing data that the consumer doesn't need or isn't authorized to see. PII stripping, field projection, redaction.
Where to do it:
- In the producer: Producer emits the canonical model or consumer-specific format. Couples the producer to its consumers.
- In the consumer: Consumer accepts the producer's format and translates internally. Couples the consumer to the producer's model.
- In a mediator: A dedicated transformation service or integration layer does the translation. Adds infrastructure but isolates both sides. This is the ESB model's core value proposition, and it's the right choice when you have many-to-many integrations with different schemas.
Content-Based Routing
What it is. Inspecting the content of a message to decide where to route it. An order message with country = "DE" goes to the EU fulfillment system. An order with country = "US" goes to the US fulfillment system.
When to use: When the routing decision depends on business data inside the message, not just metadata (topic, header, source system).
The risk: The routing logic becomes a hidden business rule. When routing rules are embedded in integration middleware, developers and product managers don't know they exist. Document routing rules as first-class business logic. Version them. Test them. Review them when business rules change.
Scatter-Gather
What it is. Send a request to multiple systems in parallel, collect all responses, and aggregate them into a single response.
Example: A price comparison that queries 5 supplier systems simultaneously and returns the best offer.
Key decisions:
- Timeout strategy. Do you wait for all responses or return after a timeout with whatever you have? Most implementations should use a timeout — one slow supplier shouldn't block the entire response.
- Partial failure. If 3 of 5 suppliers respond, is that a success or a failure? Define this up front. Most implementations treat partial results as success with degraded quality.
- Result aggregation. How do you combine the responses? Best price? Merge and deduplicate? Union? The aggregation logic is domain-specific and often more complex than the scatter or gather.
Message Enrichment
What it is. Augmenting a message with additional data from external sources before delivering it to the consumer.
Example: An order event arrives with customerId: 12345. The enricher looks up the customer's shipping address, credit tier, and communication preferences, and attaches them to the order event before forwarding it to the fulfillment system.
The tradeoff: Enrichment creates a runtime dependency on the enrichment source. If the customer service is down, enrichment fails, and either the message is delayed (queued for retry) or delivered incomplete. Decide up front which enrichments are required (block until available) and which are optional (deliver without them and let the consumer handle the gap).
Schema Mapping and Impedance Mismatch
Every pair of systems has a different mental model of the same business entities. The customer system has a Customer with addresses, preferences, and communication history. The billing system has an Account with payment methods, invoices, and credit terms. They're talking about the same person, but their models serve different purposes.
Mapping strategies:
- Field-level mapping. Direct mapping between fields.
Customer.email→Account.contactEmail. Works for simple cases, breaks when the cardinality differs (Customer has multiple emails, Account expects one). - Structural mapping. Reshaping data — flattening nested structures, combining fields, splitting records. This is where most mapping complexity lives.
- Lossy mapping. Accepting that some data cannot be mapped and will be lost in translation. This is often the right answer — the billing system doesn't need the customer's communication history. Document what's lost so future debuggers don't spend hours looking for data that was intentionally dropped.
- Versioned mappings. As schemas evolve on either side, the mapping must evolve too. Treat mappings as versioned artifacts with their own tests and release process.
6. Organizational Patterns
The architecture of your integration reflects the structure of your organization. This is not a suggestion — it's a law of nature (Conway's law). If you want to change the architecture, you must change the organization, or the organization will route around your architecture.
Conway's Law as a Design Constraint
Conway's law, restated for integration: The interfaces between your systems will mirror the communication structures between the teams that build them. If two teams don't talk to each other, their systems won't integrate well. If a team owns three systems, those systems will be tightly coupled. If an integration is owned by "everyone" (no specific team), it will be owned by no one and will rot.
Using Conway's law offensively:
- If you want loosely coupled systems, give each system to a team with clear boundaries and well-defined interfaces to other teams.
- If you want a clean integration layer, create a dedicated integration team — but understand that this team will become a bottleneck unless they build self-service platforms rather than hand-coding integrations.
- If two systems need to be tightly integrated, consider whether the teams should be merged or at least co-located (physically or in communication channels).
The anti-pattern: Designing a beautiful decoupled architecture on a whiteboard and then assigning it to an organization that doesn't match. The org structure will win. Always.
Team Topologies for Integration
Borrowed from the Team Topologies framework (Skelton & Pais), applied to enterprise integration:
Stream-aligned teams. Own a business capability end-to-end, including its integrations. The order processing team owns the order service, its API, its event publications, and its integrations with payment and fulfillment. This is the ideal for integration ownership because the team that understands the domain owns the integration logic.
Platform teams. Provide the shared integration infrastructure — the message broker, the API gateway, the CDC platform, the schema registry, monitoring and alerting. They don't own integrations; they own the tools that make integrations possible. A well-functioning platform team reduces the cognitive load on stream-aligned teams by providing self-service capabilities.
Enabling teams. Help stream-aligned teams adopt integration patterns and tools. A team of integration specialists that pairs with product teams during complex integration projects, teaches patterns, and then moves on. This is how you spread integration expertise without centralizing integration ownership.
Complicated subsystem teams. Own specific technically complex integration components — a high-performance message transformation engine, a complex protocol adapter for a legacy mainframe, a real-time data synchronization system. These exist when the technical complexity is too high for a stream-aligned team to handle alongside their domain work.
API Contracts as Team Contracts
An API between two systems is a contract between two teams. Technical API design (REST, gRPC, event schema) is the easy part. The hard part is the social contract:
- Who can change the API? The producer team? Only with consumer approval? Through a review process?
- What's the deprecation policy? How much notice do consumers get before a breaking change? Who decides what's breaking?
- What's the SLA? Availability, latency, throughput — these aren't just technical specs. They're commitments from one team to another. If the order service promises 99.9% availability to the fulfillment service, that's a commitment that affects on-call staffing, infrastructure investment, and deployment practices.
- Who debugs integration failures? When data doesn't flow correctly, whose problem is it? Define this before the first production incident, not during it.
Practical guidance: Write integration contracts as documents that both teams sign off on. Not legal documents — living documents in a shared wiki or repository. Include: endpoint/topic definitions, schema versions, SLAs, breaking change policy, escalation contacts, and a decision log of why things are the way they are. The decision log is the most valuable part — six months later, nobody will remember why the order event uses a string for quantity instead of an integer.
The Integration Team Paradox
Every enterprise eventually creates a "central integration team." This team owns the ESB, the API gateway, the iPaaS platform, or whatever the current integration middleware is. The paradox:
- If the integration team owns integrations: They become a bottleneck. Every team that needs an integration files a ticket and waits. The integration team is perpetually understaffed because they scale linearly with the number of integrations. They don't understand the domain context of what they're integrating and make mapping errors. Product teams route around them with direc
…(truncated)