Message Queuing Frameworks – Comparison Grid


We often come with requirements which are suited for integrating Messaging Frameworks in the Software Systems.
There are many messaging frameworks available in the market – some are open-source, some are paid-licensed, some provide great support, have good Community support.

In order to make an apt choice, we look out and explore different messaging frameworks based on our requirements.

This post compares few Popular Messaging Frameworks and aims to provide or equip you with enough information to make a decision on choosing the best framework as per your requirements.

COMPARISON GRID

RabbitMQ Apache Kafka AWS SQS
HA ☑ Requires some extra work and
may require 3rd party Plugins like Shovel and Federation
☑ Out of the Box (OOB) ☑ OOB
Scalable
Guaranteed Delivery ☑ Supports Consumer Acknowledgments ☑ Supports Consumer Acknowledgments ☑ Supports Consumer Acknowledgments
Durable ☑ Through Disk Nodes and Queues with extra configuration ☑ OOB ☑ Message Retention upto 14 days max and default being 4 days.
Exactly-Once Delivery ☑ Annotates a message with redelivered when
message was delivered earlier but consumer ack failed earlier.
Requires Idempotent behavior of a Consumer
☑ Dependent on Consumer behavior.

Consumer is made responsible to track Offsets (messages read so far) and store those offsets. Kafka started supporting storing offsets within Kafka itself.

It supports storing Offsets OOB through HIGH LEVEL CONSUMERS, however

Requires
Idempotent
behavior of a Consumer

☑ MessageDeDup ID and MessageGroupID attributes are used.

Requires
Idempotent
behavior of a Consumer.
FIFO supports Exactly-once while Standard Queues support Atleast-Once

Ease of Deployment ☑ For Distributed Topology,
Requires more effort and 3rd party Plugins
☑ Requires ZooKeeper ☑ Managed by AWS
Authentication Support ☑ OOB ☑ OOB ☑ OOB
Authorization aka Acl Support ☑ OOB ☑ OOB ☑ OOB
TLS Support ☑ OOB ☑ OOB ☑ OOB
Non-Bocking Producers ☑ Supports both Synchronous and Async
Performant ⭐ ⭐ Medium to High ⭐ ⭐ ⭐ ⭐ Very High ⭐ ⭐ ⭐ ⭐ Very High

FIFO: 300 tps
Standard Queues: Unlimited

Open Source  ☑  ☑
Load Balancing Across Consumers ☑ Can be done Through Consumer Groups ☑ Multiple Consumers can read from the same queue in an atomic way
Delay Queues NOT OOB NOT OOB ☑ OOB
Visibility Timeout Queues NOT OOB NOT OOB ☑ OOB
Message Dedup  ☑
Message Size Limits Upto 256 KB. AWS SDK supports storing large messages in S3 etc though.
No of Messages in a Queue ☑ No limits ☑ No Limits ☑ No Limits but

Standard Queues: 1,20,000 In-Flight Messages
FIFO: 20,000 In-Flight Messages

Details here and here

Messages are In-Flight after they have been received from the queue by a consuming component, but have not yet been deleted from the queue

Message Content Limits ☑ No Limits ☑ No Limits A message can include only XML, JSON, and unformatted text. The following Unicode characters are allowed: #x9 | #xA | #xD | #x20 to #xD7FF | #xE000 to #xFFFD | #x10000to #x10FFFF

Any characters not included in this list are rejected.

Disaster Recovery Not OOB Not OOB but simple. Replicas can be deployed across regions Not OOB and simple. Requires different Strategies to achieve it.

Advertisements

About Sunil Singhal

A human being whose dreams are tied to a Horse that will never tire
This entry was posted in Common Technical Problems, Technical and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s