Category Archives: Common Technical Problems

Common Technical Problems, Technical

Message Queuing Frameworks – Comparison Grid

1 Comment

We often come with requirements which are suited for integrating Messaging Frameworks in the Software Systems.
There are many messaging frameworks available in the market – some are open-source, some are paid-licensed, some provide great support, have good Community support.

In order to make an apt choice, we look out and explore different messaging frameworks based on our requirements.

This post compares few Popular Messaging Frameworks and aims to provide or equip you with enough information to make a decision on choosing the best framework as per your requirements.

COMPARISON GRID






RabbitMQ
Apache Kafka
AWS SQS




HA
☑ Requires some extra work and
may require 3rd party Plugins like Shovel and Federation
☑ Out of the Box (OOB)
☑ OOB




Scalable
☑
☑
☑




Guaranteed Delivery
☑ Supports Consumer Acknowledgments
☑ Supports Consumer Acknowledgments
☑ Supports Consumer Acknowledgments




Durable
☑ Through Disk Nodes and Queues with extra configuration
☑ OOB
☑ Message Retention upto 14 days max and default being 4 days.




Exactly-Once Delivery
☑ Annotates a message with redelivered when
message was delivered earlier but consumer ack failed earlier.
Requires Idempotent behavior of a Consumer
☑ Dependent on Consumer behavior.

Consumer is made responsible to track Offsets (messages read so far) and store those offsets. Kafka started supporting storing offsets within Kafka itself.

It supports storing Offsets OOB through HIGH LEVEL CONSUMERS, however

Requires 
Idempotent
 behavior of a Consumer
☑ MessageDeDup ID and MessageGroupID attributes are used.

Requires 
Idempotent
 behavior of a Consumer.
FIFO supports Exactly-once while Standard Queues support Atleast-Once




Ease of Deployment
☑ For Distributed Topology,
Requires more effort and 3rd party Plugins
☑ Requires ZooKeeper
☑ Managed by AWS




Authentication Support
☑ OOB
☑ OOB
☑ OOB




Authorization aka Acl Support
☑ OOB
☑ OOB
☑ OOB




TLS Support
☑ OOB
☑ OOB
☑ OOB




Non-Bocking Producers

☑
☑ Supports both Synchronous and Async




Performant
⭐ ⭐ Medium to High
⭐ ⭐ ⭐ ⭐ Very High
⭐ ⭐ ⭐ ⭐ Very High

FIFO: 300 tps
Standard Queues: Unlimited




Open Source
 ☑
 ☑





Load Balancing Across Consumers
☑
☑ Can be done Through Consumer Groups
☑ Multiple Consumers can read from the same queue in an atomic way




Delay Queues
NOT OOB
NOT OOB
☑ OOB




Visibility Timeout Queues
NOT OOB
NOT OOB
☑ OOB




Message Dedup


 ☑




Message Size Limits


Upto 256 KB. AWS SDK supports storing large messages in S3 etc though.




No of Messages in a Queue
☑ No limits
☑ No Limits
☑ No Limits but

Standard Queues: 1,20,000 In-Flight Messages
FIFO: 20,000 In-Flight Messages

Details here and here

Messages are In-Flight after they have been received from the queue by a consuming component, but have not yet been deleted from the queue




Message Content Limits
☑ No Limits
☑ No Limits
A message can include only XML, JSON, and unformatted text. The following Unicode characters are allowed: #x9 | #xA | #xD | #x20 to #xD7FF | #xE000 to #xFFFD | #x10000to #x10FFFF

Any characters not included in this list are rejected.




Disaster Recovery
Not OOB
Not OOB but simple. Replicas can be deployed across regions
Not OOB and simple. Requires different Strategies to achieve it.

Advertisement
.NET, Common Technical Problems, Technical

Playaround with an address of object reference variable

5 Comments

Well, rarely you will have to find an address of a variable in C#.NET. Though C#.NET allows use of pointers but anyone hardly use them. .NET base library and Compiler have done a beautiful job, abstracting the complex use of pointers, exposing ref, out keywords.

But what if you still want to playaround?
[I must warn you about this. Don’t dare to play with Pointers 🙂 Improper handling of pointers can even bring your application down]

Thankfully, there are constructs which do help in achieving what we want.

The above link is just to get a concept of Stack. What a stack is. It has nothing to do with Stack class provided in C#.NET.

static void GetAddress()
{
unsafe
{
int i = 5;
object refer = new object();
// &i gets an address of variable 'i' and * operator gets the value
Console.WriteLine("Address of i:{0},{1}" , (uint)&i, *(&i));
Console.WriteLine("Address of refer:{0},{1}", (uint)(&i - 1), *(&i - 1));
}
}

Explanation
Here, i have defined two variables, named – i and refer . Both these variables sit on STACK, however the values stored by them are treated differently.
i stores the value directly [5 in our case as per statement], and
refer stores the address of an actual object allocated on a manged heap.
This is how Stack and Heap state would be after two initalizing statements.

So, if i have an access to an address of i and knowing that refer is on Stack just after i, decrementing 1 from i address [Pointer arithmetic], i’m now pointing to the refer which is the reference to the actual object (object()) sitting on heap.

You have a stack address and thus the value stored at that location. Using Pointer Arithmatic, do what you want WITH CARE.
MS should tag POINTERS with “HANDLE WITH UTMOST CARE” 🙂

Common Technical Problems

How to Reference or use same fully qualified class name from different assemblies

Leave a comment

Two or more different assemblies can contain a type with a same name and same namespace.
What will happen if you reference those assemblies in your project?
Well, referencing won’t result into any error. 🙂
But.
How about making an object of that class?
This will fail. Compilation error will be generated because of ambiguity. Which type from which assembly should be used?
Compiler can’t make this decision on it’s own and rather i would say, it should not.
This should be done by a programmer/coder to clear his intentions to the compiler.
But How?

Solution

    Let’s say, we have two assemblies:

  • Assembly1
  • Assembly2

Both assemblies define a type, say class having fully qualified name as SomeNameSpace.SomeClassName

We have a project where we refer the above two assemblies.
In order to instantiate an object of ProjectLibrary.Class defined in Assembly1, do:

  1. Change the Aliases property of one of the references

    2. Whenever any reference is added, Aliases is set to global. Change this to your desired name, say Assembly1
    Though this can be done for both but doing it for one will automatically open doors for another assembly.

  2. Write a code statement on the top of your file: extern alias Assembly1;

  3. Lastly, change the object instantiation statement to
    Assembly1::ProjectLibrary.Class obj = new Assembly1::ProjectLibrary.Class();

    4. “::” is a Namespace Alias Qualifier in .Net.

That’s it.