RabbitMQ – An Amazing Message Broker
- Erlang Solutions Team
- 1st Dec 2022
- 11 min of reading time
In cloud architectures (or microservices), applications are broken down into smaller independent blocks that can be quickly developed, deployed, and maintained. Imagine you have a cloud architecture that has many services and many requests per second, you have to make sure that no requests are lost and your web service is always ready to receive new requests instead of locked by processing the previous request and must ensure that the services communicate with each other smoothly and efficiently.
So how do you? How can different applications communicate with each other? The answer is Message Broker!
Nowadays, there are many Message Broker software that can be listed as AWS Simple Queue Service (SQS), Apache Kafka, Apache ActiveMQ,… But the most popular among the names listed above are RabbitMQ!
So, What is a message broker? What is RabbitMQ? Why use RabbitMQ?
Your concerns will be answered by the RabbitMQ team right in the article!
What is a Message Broker?
A message broker (an integration broker or interface engine) is an intermediary module that transfers messages from sender to receiver. It is an architectural pattern for inspecting, relaying, and navigating messages; mediating between applications, simplifying communication between them, and maximizing the efficiency of splitting into smaller blocks. The main task of a Message broker is to receive messages from applications and perform some action. Let’s take a look at the sequence diagram below:
How does Message Broker work?
During the Initialize process, Service1 and Sevice2 load the proxy and register to the Broker. From there, the Broker will forward the messages to the pre-registered proxy.
The advantages can see here are:
- Service1 and Service2 don’t need to know each other and don’t need to speak in the same language. They just need to send the message to the proxy, and from here proxy will forward the message to the Broker. The Broker will take care of validating, transforming, and routing messages between Service1 and Service2.
- With this design pattern, we can set up \an asynchronous mechanism. Service1 doesn’t need to care when a message is delivered to Service2 and when Service2 finishes handling the message, everything Sevice1 should do is send the message to the Broker, Service2 will pick up the message whenever it wants.
To put it simply, a message broker is an intermediary program developed to serve the needs of easy communication between different applications. You can also understand message broker as a message broker software program.
Some highlights about RabbitMQ
RabbitMQ is an open-source message broker. In the beginning, it was used for Advanced Message Queueing Protocol (AMQP), and after that growing up to support the Streaming Text Oriented Messaging Protocol (STOMP), Message Queuing Telemetry Transport (MQTT), and other protocols.
In a word, RabbitMQ is like an intermediary message carrier or queue manager. RabbitMQ presented in Erlang language gives programmers an intermediary means to exchange data between members of the system and securely stores the data before it is pushed to another location.
It can be understood simply, as a large-scale system, the exchange of messages between components is more and more complex. Therefore, RabbitMQ was born as an effective solution in the system structure. Not only that, but the capacity of RabbitMQ is also quite light, programmers can deploy it on both fixed and cloud environments.
RabbitMQ also supports multiple Messaging protocols, develops in a distributed, federated environment, and meets the need for scalability. RabbitMQ also provides a wide range of tools for the most popular languages such as C++, C#, Java, PHP, Python,…
The most outstanding features of RabbitMQ
As the software of choice is used so much, it must contain many outstanding features:
- Interface:
RabbitMQ has a fairly simple interface, easy to use. Users can easily manage, monitor and control all problems in the programs.
- Reliability:
RabbitMQ offers a variety of features to let you trade off performance with reliability, including persistence, delivery acknowledgments, publisher confirms, and high availability.
- When a connection fails, messages may be in transit between the client and server. They may be stuck in the middle of being decoded or encoded on either side, stuck in the TCP buffer, … In such events, messages in transit will not be delivered, they will need to be retransmitted. The RabbitMQ Acknowledgements and Confirms feature will let the server and clients know when to do this.
- RabbitMQ can detect dead TCP connections with the Heartbeats feature.
- With some messaging protocols supported by RabbitMQ, applications can control the durability of queues and messages. In order to avoid losing messages, durable queue is the recommended option, and messages are published as persistent by publishers (Delivery mode property).
- In a RabbitMQ cluster, all definitions (of exchanges, bindings, users, etc) are replicated across the entire cluster. Queues may be located on a single node, or replicate their content for higher availability.
Quorum queues is a modern replicated queue type that focuses on data safety. They provide a more understandable, in some cases less resource intensive, new implementation for achieving replicated queues and high availability.
Streams is another replicated messaging data type that allows for repeatable consumption.
- Flexibility:
The message is routed through the Exchange before reaching the Queue. RabbitMQ provides some commonly used Exchange types, we can also define our own Exchange as a plugin. For more complex routing you can bind exchanges together.
RabbitMQ enables users to control the trade-offs between messages, throughput, and performance. All the messages in the queues can specify where they should be saved to a disc before their delivery. Queues in a cluster can span multiple servers while ensuring that no messages are lost in the case of a server failure.
- Multi-protocol, the multi-language feature creates a diversity of users.
- Lightweight:
RabbitMQ is lightweight and requires less than 40MB of RAM to run the application core and plugins like the Management UI.
- High availability of queues:
With queues in RabbitMQ, users can replicate it on several different machines in the same cluster. This will help to keep the messages safe even if the hardware fails.
- Traceability:
If messaging is not working properly, RabbitMQ will step in and take action. Thanks to its traceability, users can track the system’s operating status or tell if the system has any problems.
- Plugins system:
RabbitMQ also supports plugin extension through many forms. If you have the ability, you can also create these utilities yourself. For more information on how to develop a RabbitMQ Plugin, please refer to the Plugin Development Basics page.
- Commercial Services:
Support sales with training and consulting programs offered on Pivotal.
RabbitMQ’s application
RabbitMQ can be used when the web server needs to quickly respond to requests. This eliminates the need to perform resource-intensive operations while the user waits for results. RabbitMQ is also used to transport messages to many different recipients for processing or to share the load among highly loaded workers (20K+ messages/sec).
RabbitMQ can be used for:
- Applications need to support legacy protocols, such as STOMP, MQTT, AMQP, 0-9-1.
- Fine-grained control over consistency/set of guarantees on a per-message basis
- Complex routing to consumers
- Applications need multiple to publish/subscribe, point-to-point request/reply messaging capabilities.
Who is using RabbitMQ?
RabbitMQ is an open-source tool with 10.1K GitHub stars and 3.7K GitHub forks. Here’s a link to RabbitMQ’s open-source repository on GitHub.
1972 companies reportedly use RabbitMQ in their tech stacks, including Robinhood, Reddit, and Tech Stack.
Reference resources
https://github.com/rabbitmq/rabbitmq-server
