Integration Patterns Guide

Integration Fundamentals

Enterprise integration connects systems that were not originally designed to work together. Each system has its own data model, communication protocol, and operational characteristics. Integration patterns provide proven structural approaches for bridging these differences in a maintainable, observable, and fault-tolerant way.

The choice of integration pattern affects system coupling, operational complexity, and failure characteristics. Point-to-point integrations are simple to implement but create web of dependencies. Message broker architectures reduce coupling but add infrastructure complexity. Selecting the right pattern for a given integration scenario reduces long-term maintenance burden.

Point-to-Point Integration

Point-to-point integration connects two systems directly. System A sends data to System B through a dedicated integration pathway. This pattern is appropriate for integrations between two stable systems with a well-defined, unchanging data contract.

Point-to-point integrations are simple to understand and debug in isolation. They become problematic at scale: ten systems with point-to-point connections require up to 45 separate integration pathways (n*(n-1)/2). Changes to any system's API or data model require updating every connected integration. For organizations with more than three or four systems requiring interconnection, hub-and-spoke or event-driven architectures reduce maintenance overhead significantly.

Hub-and-Spoke Architecture

Hub-and-spoke architecture routes all integration traffic through a central integration layer (the hub). Each system connects to the hub rather than directly to other systems. When System A needs to send data to System B, it sends to the hub, which transforms and routes the data to System B.

This architecture centralizes transformation logic, reducing duplication and providing a single point for monitoring and audit logging. The hub becomes a single point of failure; high-availability hub configurations are required for production deployments. An iPaaS platform or an enterprise service bus (ESB) typically implements the hub in modern deployments.

Event-Driven Integration

Event-driven architectures decouple systems by routing communication through an event stream or message queue. A system publishes events when something changes (order placed, record updated, payment received). Subscribing systems consume events independently and process them at their own pace.

Message Queues

Message queues buffer events between publishers and consumers. The queue guarantees delivery even if the consumer is temporarily unavailable. Consumers process messages at their own rate, preventing fast-producing systems from overwhelming slow-consuming systems. Message queues provide at-least-once delivery guarantees; consumer workflows must be idempotent to handle duplicate message delivery.

Event Streaming

Event streaming platforms retain a ordered log of events for a configurable retention period. Consumers read from any position in the log, enabling replay of past events, independent consumption speeds across multiple consumers, and recovery from consumer failures without message loss. Event streaming is appropriate for integrations where multiple systems need to process the same event independently.

API Integration Patterns

Request-Response

A workflow calls an API endpoint and waits for a synchronous response. This pattern is straightforward but creates temporal coupling: the calling workflow waits idle while the API processes. HTTP timeouts must be set conservatively. For operations that take more than a few seconds, asynchronous patterns are preferable.

Polling

When a target system does not support webhooks or event emission, a workflow polls at regular intervals to detect new or changed records. Polling introduces latency proportional to the polling interval and generates API load whether or not data has changed. Where possible, polling should be replaced with webhook-based notification as the target system's capabilities improve.

Webhook-Based Integration

The source system pushes notification payloads to a registered URL when relevant events occur. Webhook payloads typically contain a minimal event descriptor plus a record identifier; the receiving workflow fetches full record details through a subsequent API call. Webhook integration reduces latency and API call overhead compared to polling. Security requirements for webhook endpoints include payload signature verification to prevent unauthorized triggering.

API Orchestration

An orchestration workflow coordinates calls to multiple APIs to complete a single business transaction. The orchestrator calls System A, uses the response to build a request for System B, merges both responses, and delivers a combined result. Orchestration centralizes transaction logic in the workflow layer, making the overall flow visible and testable. Orchestration workflows are more complex than point-to-point integrations and require explicit error handling for each API call in the sequence.

Data Transformation Patterns

Canonical Data Model

A canonical data model defines a common data format for the integration layer. Each system transforms data to and from the canonical format rather than to and from every other system's format. With n systems, a canonical model requires 2n transformations instead of n*(n-1). The canonical model requires upfront design investment and ongoing governance as source system schemas evolve.

Transformation Step Design

Transformation steps map source fields to target fields, apply calculations, filter records, and enrich data from reference sources. Keep transformation logic simple and deterministic. Complex multi-step transformations should be decomposed into discrete, testable transformation steps. Embed field-level validation in transformation steps to catch schema mismatches at transformation time rather than at the target system boundary.

Error Handling in Integration Workflows

Integration failures occur at network boundaries and API endpoints. Design error handling at three levels: transient failure recovery (retry with exponential backoff), permanent failure routing (dead-letter queues, error notifications, manual review queues), and data quality rejection handling (validation failures, schema mismatches that require source correction). Document recovery procedures for each error category so operations teams can resolve failures without requiring the original developer's involvement.