Queues and pub-sub

Question 1

Middleware

Answer 1

a system layer that provides commonly-needed functionality as a service so that developers can focus on key functional or logical system components.

Middleware can be just a portion of the code or an entire server cluster. The sorts of common intermediary functions middleware provide include translation, transaction processing, metrics monitoring, and message passing.

Question 2

Messaging-oriented middleware

Answer 2

Any complex system will have different components, possibly running entirely different hardware and software, that need to be able to communicate with each other. Messaging-oriented middleware (MOM) enables this communication, much like the post office enables people to send each other letters. Producers hand off packets of data called messages to the MOM which makes sure it’s delivered to the correct consumers.

Message passing allows components to communicate asynchronously. In other words, a producer can send messages independently of the state of the consumer. If the consumer is too busy or offline, a MOM will make sure the messages are delivered once the consumer becomes available again.

Asynchronicity enables system components to be decoupled from each other. This adds resilience because, when one component fails, the others can continue functioning normally. It also adds data integrity because successful message passing isn’t dependent on the producer and consumer being responsive at the same time.

The software that implements a MOM can be called a message broker. Message brokers may implement just one, or several different kinds of message passing including both queues and pub-sub

Question 3

Message queues

Answer 3

Message queues are a kind of messaging-oriented middleware where producers push new messages to a named First-In, First-Out (FIFO) queue, which consumers can then pull from.

Message queues are also called point-to-point messaging because there is a one-to-one relationship between a message’s producer and consumer. There can be many producers and consumers using the same queue, but any particular message will only have one producer and one consumer.

Question 4

Pub-sub

Answer 4

Publisher-subscriber. a kind of messaging-oriented middleware that pushes a producer’s newly “published” messages based on a “subscription” of the consumer’s preferences.

There is a one-to-many relationship between publishers and subscribers, meaning any number of subscribers can get a copy of a message, but there’s only one publisher of that message. Pub-sub doesn’t guarantee message order, just that consumers will only see messages that they’ve subscribed to.

Subscriptions can be filtered by both topic and content. Topics are any category defined by the publisher, and content is any category defined by a subscriber. It’s up to the message broker to accept and manage these filters properly.

Question 5

Which to use

Answer 5

Whether to use queues or pub-sub depends mostly on how many message consumers the system has. If a message needs to have only one consumer, then the message queue is the right approach. If a message needs to have possibly many consumers that get a copy, the pub-sub approach is best.

Implementations also differ in the particular message-passing guarantees they provide. Some feature variations to note are:

Persistence: the messages are saved to persistent storage so they can be recovered in case the message broker itself goes down.
Replays: the messages are stored even after they are consumed so a service can “replay” the message stream in failure cases to recover properly.
Ordering: the messages always arrive to the consumer in a particular order.