RabbitMQ Flashcards
What are the alternatives to RabbitMQ
ActiveMQ
Kafka
zeroMQ
Qpid
Why RabbitMQ
- RabbitMQ Supports AMQP, STOMP and MQTT protocols.
- Clustering is very simple because of Erlang
- Far more reliable and crash resistant than its competitors
- Very easy to install and use
How do producers/consumers connect to RabbitMQ?
They connect to RabbitMQ by creating a TCP connection. An AMQP channel is created once the TCP connection is open. This channel is virtual connection inside real TCP connection, and all AMQP commands are sent over this channel. Each channel is assigned a unique ID. Multiple channels can be opened on a single TCP connection.
How message is routed between consumers and producers?
There are three parts to any successful routing of AMQP messages – exchange, queue and binding.
Exchanges are where producers publish their message, queues are where messages end up & received by consumers and bindings are how messages are routed from exchange to queues.
How rabbitMQ queue works?
Queues are like named mail-box. They are where messages end up and wait to be consumed. Consumers receive messages from a queue in one of two ways –
• Push model: Consumers automatically receive messages as they arrive in the queue. (Basic.Consume)
• Pull model: Consumers requests a single message from queue. Once that is processed, the next message is requested by the consumer. (Basic.Get)
If multiple consumers have subscribed to the queue, they are served in round-robin fashion. Each message is sent to only one consumer subscribed to the queue. A message is removed from queue on successful acknowledgement from the consumer. Consumer can acknowledge using basic.ack command or by setting auto_ack parameter to true when it subscribes to the queue. If acknowledgement is not received, then RabbitMQ sends the message to next subscriber. The consumer can reject a message either by disconnecting from RabbitMQ server without acknowledging the message or call basic.reject command. If requeue parameter of reject command is set to true, RabbitMQ will redeliver the message to next subscribed consumer. You can also discard a message by simply acknowledging it.
What are important properties of a queue
Important properties of queue are
• Name – name used by consumer to subscribe to it. If not specified RabbitMQ will generate some random name.
• Exclusive – Private queue that can only be consumed by your application
• Auto-delete – automatically deleted when the last consumer unsubscribes. If you need a temporay queue used by only one consumer, combine auto-delete with exclusive. When consumer disconnects, the queue will be deleted
• x-expires: Queue is removed after specified number of miliseconds
• x-max-length: Maximum message count for a queue
• x-message-ttl: Message expiration in milliseconds.
What happens if message is published into an exchange but queue is not declared at that time?
Message that get published into exchange but have no queue to be routed to are discarded by rabbit. So, if you cannot afford your messages to be black holed, both producer and consumer should attempt to create the queue.
How can you load balance message processing in multiple consumers
Queues are fundamental block of AMQP messaging. They are perfect for load balancing. Just attach a bunch of consumers and let RabbitMQ round-robin incoming messages evenly among them.
What are different types of queues
- Temporary queues – Queues are deleted when the last consumer disconnect. (auto_delete = True)
- Exclusive – Only a single consumer can access the queue. Automatically deleted when the channel that queue was created on is closed.
- Expiring – Automatically expiring queues means queues are deleted if it is unused for some length of time. ‘x-expires’ argument used while declaring the queue.
- Permanent - Permanent queues persist across server restart. Queue durability is different than message durability. The messages can be auto expired using “expires” attributes set in the message properties. ‘x-message-ttl’ setting of queue enforces maximum age for all the messages in the queue. ‘x-max-length’ argument specifies maximum length of the queue. If this capacity is reached, RabbitMQ will drop messages from front of queue as new messages are added.
How does a message reach a queue
A message is sent to exchange by publisher. Message has payload and routing key. A queue when created is bound to an exchange by routing keys. Message routing key is matched against the queue binding (i.e. routing key specified by the queue) and message is delivered to the queue whose binding matches. So, the broker routes messages from exchange to queues based on routing keys.
What is direct exchange
Direct exchange is used to deliver messages to specific targets. It uses string equality when checking the bindings with routing keys, no pattern matching (like regular expressions) are supported. Multiple queues can be bound to the direct exchange. There is a default direct exchange which has empty string as its name. When a queue is declared, it will bind to this default exchange.
Send a message to the default exchange
Channel.basic_publish($msg, ‘’, ‘queue_name’)
Declare your own exchange if default is not enough. It’s a good choice for routing RPC reply messages.
What is fanout exchange
Message sent to fanout exchange will be delivered to all the queues attached to it. RabbitMQ does not evaluate routing keys associated with messages as all are delivered to all the queues.
What is topic exchange
Route Message to any queue bound with matching routing key. A message can be sent to multiple queues.
Channel.basic_publish($msg, ‘logs_exchange’, ‘error.msg-inbox’)
Channel.queue_bind(‘msg-inbox-errors’, ‘logs_exchange’, ‘error.msg-inbox)
In above example, message is published to ‘logs_exchange’ with routing key as ‘error.msg-inbox’. It will get delivered to the queue ‘msg-inbox-errors’ since this queue is bound to the same ‘logs_exchange’ and the queue binding ‘error.msg-inbox’ matches the routing key of the published message.
This behavior is so far like direct exchange. But if you declare additional queue like
Channel.queue_bind(‘all-logs’, ‘logs_exchange’, ‘#’)
Then all the messages sent to logs_exchange will be sent to ‘all-logs’ queue. # indicates match-all. You can use wild cards in the queue binding. For example
Channel.queue_bind(‘inbox-logs’, ‘logs_exchange’, ‘*.msg-inbox)
All the messages whose routing key matches ‘*.msg-inbox’ would be sent to ‘inbox-logs’ queue. Here * means all characters and . acts as a separator.
Namespaced (combinations of dots in the name of the key) routing key is a good choice for future proofing an application. Topic exchange can emulate behavior of both direct and fanout exchange. Direct using full routing keys instead of pattern matching and fanout using # routing keys.
If an exchange is declared with ‘auto-delete’ set to true, then that exchange would be automatically deleted when the last consumer connected to it is disconnected.
what is exchange to exchange routing
This is supported using Exchange.Bind rpc command.
What is consistent hashing exchange
Distribute messages to queues on different physical servers
What are different types of message deliveries supported by RabbitMQ?
RabbitMQ provides different guarantees of message delivery. Each delivery option comes with some performance impact.
• No guarantee: This is the fastest way to deliver the message. There is no guarantee that the message would be delivered as publisher does not have any way to know about it.
• Mandatory argument in Basic.Publish command – The argument tells RabbitMQ that the non-routable message should be sent back to publisher vai Basic.Return RPC command. Basic.Return is an asynchronous call from RabbitMQ.
• Publisher confirms: In this mechanism every published message is responded with Basic.Ack or Basic.Nack response, asynchronously. The channel needs to be put in “publisher confirms” mode and its not a setting at exchange or queue level OR a message property. When channel is put in publisher confirms mode, underneath publisher first sends Confirm.Select RPC to RabbitMQ and wait for Confirm.SelectOk. Every message published on the channel will be assigned a unique ID. Once the message has been delivered to all the queues that have binding matching the message’s routing key, the channel will issue a publisher confirm to the producer application, containing the message’s unique ID. This lets producer know that the message has been safely queued at all the destinations. The message ID is unique to a channel. So once channel is closed you won’t be able to track the status of any outstanding publisher confirms for the messages published on that channel.
• Alternate exchange for unrouted messages – Setup an exchange to which unrouted messages will be sent to. This exchange is setup while setting up primary exchange using ‘add-alternate-exchange’ parameter to Exchange.Declare command. A queue can be bound to alternate exchange that will receive all unroutable messages (This queue is called dead letter queue).
• Transactions – Transaction class ‘Tx’ provides a way in which messages can be published to RabbitMQ in batches and then committed to a queue or rolled back. To begin transaction, publisher sends Tx.Select RPC to RabbitMQ and RabbitMQ responds back with Tx.SelectOk. Once a transaction is opened, publisher can send one or more messages to RabbitMQ. RabbitMQ responds with Basic.Return response if there is any error while routing the messages. Since Basic.Return is asynchronous, RabbitMQ ensures that this is sent before
Tx.CommitOk response.
• HA-queues – allows queues to have redundant copies across multiple servers. HA-queues requires clustered RMQ environment. When message is published into a queue, it is sent to each server in the cluster. Once message is consumed from any node in the cluster, all copies of message will be immediately removed from the other nodes.
• Persistent messages – Set delivery_mode property of message to 2 and such messages will be persisted on the disk. Such message can survive crash of AMQP processes. So, for a message that is in flight inside RabbitMQ to survive a crash, a message must
o Have its delivery mode option set to 2.
o Be published into durable exchange.
o Arrive in a durable queue.
Note that persistent messaging impacts performance of RabbitMQ as persistent messages are written to disk in a log file and removed from the log file once they are consumed by the consumer.
• Queues and exchange do not survive reboot because of a property on every queue and exchange called ‘durable’ whose default value is false. It is set to true then the queues and exchanges gets recreated on the reboot of RabbitMQ server.
What is the use of app-id message property?
Useful for defining the application publishing the message
What is the user of correlation-id message property?
If the message is in reference to some other message or uniquely identifiable item, the correlation-id is a good way to indicate what the message is referencing to.
What is ‘delivery-mode’ message property ?
A value of 1 tells RabbitMQ it can keep the message in memory; 2 indicates it should also write it to disk.
What is ‘expiration’ message property?
An epoch or Unix timestamp value as a text string that indicates when the message should expire
what is ‘message-id’ message property?
A unique identifier such as a UUID that your application can use to identify the message.
what is ‘timestamp’ message property?
An epoch or Unix timestamp value that can be used to indicate when the message was created.
what is ‘user-id’ message property?
A free-form string that, if used, RabbitMQ will validate against the connected user and drop messages if they match.
What are rabbitMQ credits?
RabbitMQ stops accepting messages from publishers if too many messages are sent to RabbitMQ. This is done using notion of credits. If all credits are expired, then RabbitMQ stops accepting messages from such publisher and sends him Connection.Blocked command. Connection.Unblocked is sent when the block is removed.
