RabbitMQ

Question 1

What are the alternatives to RabbitMQ

Answer 1

ActiveMQ
Kafka
zeroMQ
Qpid

Question 2

Why RabbitMQ

Answer 2

• RabbitMQ Supports AMQP, STOMP and MQTT protocols.
• Clustering is very simple because of Erlang
• Far more reliable and crash resistant than its competitors
• Very easy to install and use

Question 3

How do producers/consumers connect to RabbitMQ?

Answer 3

They connect to RabbitMQ by creating a TCP connection. An AMQP channel is created once the TCP connection is open. This channel is virtual connection inside real TCP connection, and all AMQP commands are sent over this channel. Each channel is assigned a unique ID. Multiple channels can be opened on a single TCP connection.

Question 4

How message is routed between consumers and producers?

Answer 4

There are three parts to any successful routing of AMQP messages – exchange, queue and binding.
Exchanges are where producers publish their message, queues are where messages end up & received by consumers and bindings are how messages are routed from exchange to queues.

Question 5

How rabbitMQ queue works?

Answer 5

Queues are like named mail-box. They are where messages end up and wait to be consumed. Consumers receive messages from a queue in one of two ways –
• Push model: Consumers automatically receive messages as they arrive in the queue. (Basic.Consume)
• Pull model: Consumers requests a single message from queue. Once that is processed, the next message is requested by the consumer. (Basic.Get)
If multiple consumers have subscribed to the queue, they are served in round-robin fashion. Each message is sent to only one consumer subscribed to the queue. A message is removed from queue on successful acknowledgement from the consumer. Consumer can acknowledge using basic.ack command or by setting auto_ack parameter to true when it subscribes to the queue. If acknowledgement is not received, then RabbitMQ sends the message to next subscriber. The consumer can reject a message either by disconnecting from RabbitMQ server without acknowledging the message or call basic.reject command. If requeue parameter of reject command is set to true, RabbitMQ will redeliver the message to next subscribed consumer. You can also discard a message by simply acknowledging it.

Question 6

What are important properties of a queue

Answer 6

Important properties of queue are
• Name – name used by consumer to subscribe to it. If not specified RabbitMQ will generate some random name.
• Exclusive – Private queue that can only be consumed by your application
• Auto-delete – automatically deleted when the last consumer unsubscribes. If you need a temporay queue used by only one consumer, combine auto-delete with exclusive. When consumer disconnects, the queue will be deleted
• x-expires: Queue is removed after specified number of miliseconds
• x-max-length: Maximum message count for a queue
• x-message-ttl: Message expiration in milliseconds.

Question 7

What happens if message is published into an exchange but queue is not declared at that time?

Answer 7

Message that get published into exchange but have no queue to be routed to are discarded by rabbit. So, if you cannot afford your messages to be black holed, both producer and consumer should attempt to create the queue.

Question 8

How can you load balance message processing in multiple consumers

Answer 8

Queues are fundamental block of AMQP messaging. They are perfect for load balancing. Just attach a bunch of consumers and let RabbitMQ round-robin incoming messages evenly among them.

Question 9

What are different types of queues

Answer 9

• Temporary queues – Queues are deleted when the last consumer disconnect. (auto_delete = True)
• Exclusive – Only a single consumer can access the queue. Automatically deleted when the channel that queue was created on is closed.
• Expiring – Automatically expiring queues means queues are deleted if it is unused for some length of time. ‘x-expires’ argument used while declaring the queue.
• Permanent - Permanent queues persist across server restart. Queue durability is different than message durability. The messages can be auto expired using “expires” attributes set in the message properties. ‘x-message-ttl’ setting of queue enforces maximum age for all the messages in the queue. ‘x-max-length’ argument specifies maximum length of the queue. If this capacity is reached, RabbitMQ will drop messages from front of queue as new messages are added.

Question 10

How does a message reach a queue

Answer 10

A message is sent to exchange by publisher. Message has payload and routing key. A queue when created is bound to an exchange by routing keys. Message routing key is matched against the queue binding (i.e. routing key specified by the queue) and message is delivered to the queue whose binding matches. So, the broker routes messages from exchange to queues based on routing keys.

Question 11

What is direct exchange

Answer 11

Direct exchange is used to deliver messages to specific targets. It uses string equality when checking the bindings with routing keys, no pattern matching (like regular expressions) are supported. Multiple queues can be bound to the direct exchange. There is a default direct exchange which has empty string as its name. When a queue is declared, it will bind to this default exchange.

Send a message to the default exchange
Channel.basic_publish($msg, ‘’, ‘queue_name’)

Declare your own exchange if default is not enough. It’s a good choice for routing RPC reply messages.

Question 12

What is fanout exchange

Answer 12

Message sent to fanout exchange will be delivered to all the queues attached to it. RabbitMQ does not evaluate routing keys associated with messages as all are delivered to all the queues.

Question 13

What is topic exchange

Answer 13

Route Message to any queue bound with matching routing key. A message can be sent to multiple queues.

Channel.basic_publish($msg, ‘logs_exchange’, ‘error.msg-inbox’)
Channel.queue_bind(‘msg-inbox-errors’, ‘logs_exchange’, ‘error.msg-inbox)

In above example, message is published to ‘logs_exchange’ with routing key as ‘error.msg-inbox’. It will get delivered to the queue ‘msg-inbox-errors’ since this queue is bound to the same ‘logs_exchange’ and the queue binding ‘error.msg-inbox’ matches the routing key of the published message.
This behavior is so far like direct exchange. But if you declare additional queue like

Channel.queue_bind(‘all-logs’, ‘logs_exchange’, ‘#’)

Then all the messages sent to logs_exchange will be sent to ‘all-logs’ queue. # indicates match-all. You can use wild cards in the queue binding. For example

Channel.queue_bind(‘inbox-logs’, ‘logs_exchange’, ‘*.msg-inbox)

All the messages whose routing key matches ‘*.msg-inbox’ would be sent to ‘inbox-logs’ queue. Here * means all characters and . acts as a separator.
Namespaced (combinations of dots in the name of the key) routing key is a good choice for future proofing an application. Topic exchange can emulate behavior of both direct and fanout exchange. Direct using full routing keys instead of pattern matching and fanout using # routing keys.

If an exchange is declared with ‘auto-delete’ set to true, then that exchange would be automatically deleted when the last consumer connected to it is disconnected.

Question 14

what is exchange to exchange routing

Answer 14

This is supported using Exchange.Bind rpc command.

Question 15

What is consistent hashing exchange

Answer 15

Distribute messages to queues on different physical servers

Question 16

What are different types of message deliveries supported by RabbitMQ?

Answer 16

RabbitMQ provides different guarantees of message delivery. Each delivery option comes with some performance impact.

• No guarantee: This is the fastest way to deliver the message. There is no guarantee that the message would be delivered as publisher does not have any way to know about it.
• Mandatory argument in Basic.Publish command – The argument tells RabbitMQ that the non-routable message should be sent back to publisher vai Basic.Return RPC command. Basic.Return is an asynchronous call from RabbitMQ.
• Publisher confirms: In this mechanism every published message is responded with Basic.Ack or Basic.Nack response, asynchronously. The channel needs to be put in “publisher confirms” mode and its not a setting at exchange or queue level OR a message property. When channel is put in publisher confirms mode, underneath publisher first sends Confirm.Select RPC to RabbitMQ and wait for Confirm.SelectOk. Every message published on the channel will be assigned a unique ID. Once the message has been delivered to all the queues that have binding matching the message’s routing key, the channel will issue a publisher confirm to the producer application, containing the message’s unique ID. This lets producer know that the message has been safely queued at all the destinations. The message ID is unique to a channel. So once channel is closed you won’t be able to track the status of any outstanding publisher confirms for the messages published on that channel.
• Alternate exchange for unrouted messages – Setup an exchange to which unrouted messages will be sent to. This exchange is setup while setting up primary exchange using ‘add-alternate-exchange’ parameter to Exchange.Declare command. A queue can be bound to alternate exchange that will receive all unroutable messages (This queue is called dead letter queue).
• Transactions – Transaction class ‘Tx’ provides a way in which messages can be published to RabbitMQ in batches and then committed to a queue or rolled back. To begin transaction, publisher sends Tx.Select RPC to RabbitMQ and RabbitMQ responds back with Tx.SelectOk. Once a transaction is opened, publisher can send one or more messages to RabbitMQ. RabbitMQ responds with Basic.Return response if there is any error while routing the messages. Since Basic.Return is asynchronous, RabbitMQ ensures that this is sent before
Tx.CommitOk response.
• HA-queues – allows queues to have redundant copies across multiple servers. HA-queues requires clustered RMQ environment. When message is published into a queue, it is sent to each server in the cluster. Once message is consumed from any node in the cluster, all copies of message will be immediately removed from the other nodes.
• Persistent messages – Set delivery_mode property of message to 2 and such messages will be persisted on the disk. Such message can survive crash of AMQP processes. So, for a message that is in flight inside RabbitMQ to survive a crash, a message must
o Have its delivery mode option set to 2.
o Be published into durable exchange.
o Arrive in a durable queue.
Note that persistent messaging impacts performance of RabbitMQ as persistent messages are written to disk in a log file and removed from the log file once they are consumed by the consumer.
• Queues and exchange do not survive reboot because of a property on every queue and exchange called ‘durable’ whose default value is false. It is set to true then the queues and exchanges gets recreated on the reboot of RabbitMQ server.

Question 17

What is the use of app-id message property?

Answer 17

Useful for defining the application publishing the message

Question 18

What is the user of correlation-id message property?

Answer 18

If the message is in reference to some other message or uniquely identifiable item, the correlation-id is a good way to indicate what the message is referencing to.

Question 19

What is ‘delivery-mode’ message property ?

Answer 19

A value of 1 tells RabbitMQ it can keep the message in memory; 2 indicates it should also write it to disk.

Question 20

What is ‘expiration’ message property?

Answer 20

An epoch or Unix timestamp value as a text string that indicates when the message should expire

Question 21

what is ‘message-id’ message property?

Answer 21

A unique identifier such as a UUID that your application can use to identify the message.

Question 22

what is ‘timestamp’ message property?

Answer 22

An epoch or Unix timestamp value that can be used to indicate when the message was created.

Question 23

what is ‘user-id’ message property?

Answer 23

A free-form string that, if used, RabbitMQ will validate against the connected user and drop messages if they match.

Question 24

What are rabbitMQ credits?

Answer 24

RabbitMQ stops accepting messages from publishers if too many messages are sent to RabbitMQ. This is done using notion of credits. If all credits are expired, then RabbitMQ stops accepting messages from such publisher and sends him Connection.Blocked command. Connection.Unblocked is sent when the block is removed.

Question 25

What is dead letter exchange?

Answer 25

It is like normal exchange, but it declared to be used as an exchange for rejected messages. Upon rejecting a message that is not requeued, RabbitMQ will route the message to the change specified in queues ‘x-dead-letter-exchange’ argument. Dead letter exchange is not same as alternate exchange.

Question 26

What are mirrored queues?

Answer 26

Primary copy of a message of the mirrored queue within a RabbitMQ cluster are located on a single node (the node on which the queue was declared) but it has slave copies on other nodes in the cluster. This contrasts with exchanges and bindings, which can always be on all nodes.

Each mirrored queue consists of one master and one or more mirrors. Each queue has its own master node. All operations for a given queue are first applied on the queue’s master node and then propagated to mirrors. This involves enqueueing publishes, delivering messages to consumers, tracking acknowledgements from consumers and so on.

Messages published to the queue are replicated to all mirrors. Consumers are connected to the master regardless of which node they connect to, with mirrors dropping messages that have been acknowledged at the master. Queue mirroring therefore enhances availability but does not distribute load across nodes (all participating nodes each do all the work).
If the node that hosts queue master fails, the oldest mirror will be promoted to the new master if it synchronized

Question 27

Can rabbitMQ used for communication across data centers?

Answer 27

No.

For LAN environments, clusters provide a solid platform but clusters are not meant for high-latency networks such as WANs or the internet. To connect RabbitMQ nodes across WANs or the internet, RabbitMQ comes with two plugins – federation plugin and shovel plugin.

Question 28

What is federation plugin?

Answer 28

Federation plugin provides two different ways to get messages from one cluster to another. It provides a way to transparently relay messages from the upstream node where they were originally published to the downstream node.
• Using a federated exchange, messages published to an exchange in another RabbitMQ server or cluster are automatically routed to bound exchanges and queues on the downstream host.
• Federated queues provide a way to target the messages in single queue instead of exchange.

Question 29

What is the use of Shovel plugin?

Answer 29

Shovel plugin helps to reliably and continually move messages from a source (e.g. a queue) in one broker to a destination in another broker (e.g. an exchange). The source and destination can be on the same broker (typically in different vhosts) or distinct brokers. Shovel is basically a simple pump which connects to the source and destination broker, consumes messages from queue and re-publishes each message to the destination broker using, be default, the original exchange name and the routing_key when applicable.
Advantages of shovel
Loose coupling - A shovel can move messages between brokers (or clusters) in different administrative domains.
WAN-friendly - The Shovel plugin uses client connections to communicate between brokers and is designed to tolerate intermittent connectivity without message loss.

Question 30

What are the advantages of RabbitMQ

Answer 30

• De-couples request from processing.
• Stores messages for consumers who are not alive
• Messages are routed based on tags.
• Messages are not set structure and can even send binary data.
• Hides sender and receiver from each other
• Messages can be routed in one-to-many or one-to-one fashion. In one-to-many, the message can either be broadcasted or can be sent selectively.
• Quite simple to add new processing capacity to your application. Just add more consumers and they will start getting messages from the queue in round-robin fashion.
• Rabbit supports web based management plugin which can be used to manage and configure rabbit. So instead of rabbitctl you can use web based plugin for managing rabbit.
• Rabbit also supports REST APIs which can be used to automate management and configuration of rabbit using tools such as chef, puppet etc.
• Rabbit MQ can be enhanced by adding plugins to it. By default, Rabbit supports AMQP protocol. You can achieve following by writing plugins to rabbit
o Support for protocols other than AMQP (e.g. STOMP)
o Different authentication mechanism (LDAP, HTTPP, custom database)
o Message replication
o New exchange and routing algorithm
o Message logging and auditing