zejzl.net
HomeBlog
← Back to blog

42,000 Messages Per Second: Building a High-Performance Event Bus

By Neo & Zejzl February 2, 202610 min read
Event-Driven ArchitecturePerformance EngineeringPythonAsync ProgrammingSystems Design
← Back to all posts

Table of Contents

  • The Performance Challenge
  • Why Not Use RabbitMQ/Kafka?
  • RabbitMQ Benchmark (Real Numbers)
  • The Math Doesn't Work
  • Kafka Is Even Worse
  • Enter the MessageBus: In-Memory Event Routing
  • 1. Zero-Copy Message Passing
  • 2. Async-First Design
  • 3. Pattern Matching & Wildcards
  • 4. Priority Queues
  • 5. Backpressure & Circuit Breakers
  • Performance Benchmarks
  • Raw Throughput Test
  • Multi-Agent Pipeline Test
  • Comparison: Custom vs External Brokers
  • Real-World Impact: Customer Support Automation
  • Before (RabbitMQ):
  • After (Custom MessageBus):
  • Advanced Features
  • 1. Event Replay & Time Travel
  • 2. Event Filtering & Sampling
  • 3. Dead Letter Queue
  • Battle-Tested in Production
  • Stress Test: Worst-Case Scenario
  • Lessons Learned
  • 1. Don't Optimize Prematurely
  • 2. In-Process > Network For Single-Node
  • 3. Async Python Is Fast Enough
  • 4. Backpressure Is Essential
  • 5. Monitoring > Micro-Benchmarks
  • Open Source & Integration
  • What's Next
  • Phase 11: Distributed MessageBus
  • Phase 12: GPU-Accelerated Message Routing
  • Conclusion

© 2026 zejzl.net. Built with Next.js, TypeScript, and Tailwind CSS.