Context and Problem
Messaging and API systems often handle large payloads, which can impact performance:
- Large messages slow down processing and increase memory usage
- Network latency increases when transmitting large payloads
- Message brokers and APIs have payload size limitations
Solution
The Claim-Check pattern offloads large payloads to external storage:
- Store large data objects in an external storage system
- Send a lightweight reference (claim check) through the message queue or API
- The receiving service retrieves the full data from storage when needed
Benefits
- Improved Performance
- Reduces message size, enabling faster processing
- Reduced Memory Usage
- Minimizes resource consumption in message queues
- Scalability
- Allows handling of large data efficiently
Trade-offs
- Increased Complexity
- Requires managing external storage and references
- Latency Risk
- Adds retrieval time when accessing stored data
- Storage Overhead
- Requires additional infrastructure for data persistence
Issues and Considerations
- Data Consistency
- Ensuring stored data remains available and synchronized
- Security
- Protecting stored data from unauthorized access
- Storage Lifecycle
- Managing data retention and cleanup
When to Use This Pattern
- Your system handles large messages or payloads
- You need to optimize message queues and APIs for efficiency
- You want to minimize network latency when transmitting data