Redis Streams provide a persistent, append-only log structure — ideal for event sourcing, message queues, and activity feeds without the operational complexity of Kafka.
Core Concepts
- Stream: append-only log of messages (like a Kafka topic partition)
- Entry ID:
milliseconds-sequence(e.g.,1717200000000-0) — auto-generated - Consumer Group: multiple consumers share work with at-least-once delivery
- PEL: Pending Entry List — messages delivered but not ACKed
Produce Messages
bash
# XADD appends to stream, returns entry ID
XADD orders * user_id 123 amount 49.99 status pending
# Returns: 1717200000000-0
# With explicit max length (cap at 10K entries)
XADD orders MAXLEN ~ 10000 * user_id 456 amount 99.00 status pendingCreate Consumer Group
bash
# Create group starting from all new messages
XGROUP CREATE orders order-processors $ MKSTREAM
# Or start from beginning
XGROUP CREATE orders order-processors 0Consume and Acknowledge
bash
# Read up to 10 new messages for consumer 'worker-1'
XREADGROUP GROUP order-processors worker-1 COUNT 10 BLOCK 2000 STREAMS orders >
# Process messages, then acknowledge
XACK orders order-processors 1717200000000-0 1717200000001-0Handle Stale Pending Messages
bash
# Check pending messages older than 30 seconds
XPENDING orders order-processors - + 10
# Claim stale message from dead consumer
XCLAIM orders order-processors worker-2 30000 1717200000000-0
# Redis 7+: auto-claim
XAUTOCLAIM orders order-processors worker-2 30000 0-0 COUNT 10Stream Info and Monitoring
bash
XLEN orders # total entries
XINFO STREAM orders # stream metadata
XINFO GROUPS orders # consumer group details
XINFO CONSUMERS orders order-processors # per-consumer statsPython Producer/Consumer Example
python
import redis
r = redis.Redis(host='localhost', port=6379)
# Produce
r.xadd('orders', {'user_id': '123', 'amount': '49.99'})
# Consume
messages = r.xreadgroup(
'order-processors', 'worker-1',
{'orders': '>'},
count=10, block=2000
)
for stream, entries in messages:
for entry_id, data in entries:
# process data
r.xack('orders', 'order-processors', entry_id)Key Takeaways
- Use
XADD ... MAXLEN ~to cap stream size and control memory - Consumer groups provide at-least-once delivery with the PEL tracking unacked messages
- Use
XAUTOCLAIMto recover messages from crashed consumers - Redis Streams are a great lightweight alternative to Kafka for moderate-throughput use cases
JusDB Can Help
Redis Streams simplify event-driven architectures. JusDB can design your streaming pipeline and consumer group topology.