Connecting Your Services Together Flashcards

1
Q

What is a message?

A
  • Message contains raw data
  • A message contains the data itself, not a reference
  • The sending component expects the destination component to process the mesage in a certain way. The integrity of the overall system may depend on both the send and reciever doing a specifc job
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are events?

A
  • An event is a lightweight notification that indicates that something happened.
  • Event can be sent to multiple receivers (or none at all)
  • Events a often intended to “fan out” or have a large number of subscribers for each publisher
  • The publisher of the event has no expectation about the action of receiving component takes
  • Some events are discrete units and unrealted to other events
  • Some events are part of a related and ordered series
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is one of the main consideration in choosing between messages or events?

A

Does the sending ocmponent expect the communication to be processed in a particular way by the destination component?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the two Azure services that offer messaging services?

A
  • Azure Storage Queues
  • Azure Service Bus
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is an Azure Queue Storage?

A

is a service to store large number of messages that can securely accessed from anywhere in the world using a simple REST-based interfaces

They can contain millions of messages, limited only by the capacity of the storage account that owns it.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is an Azure Service Bus

A

is a message broker system intended for enterprise applications.
Apps that use this service may have multiple communication protocols, data contracts, higher security requirements and can include both cloud and on premise services.

Service Bus is built on top of the dedicated messaging infrastructure designed for these scenarios.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are Azure Service Bus Topics?

A

They’re like queues but can have multiple subscribers.

i.e. A topic may be “Listen” for a music app. There are two subscribers, ListenHistory and AddToFavorites. Each subscriber receives its own copy of the message to process.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are the benefits of queues

A

Increase reliability
* message can stay in queue until the conmponent is available to process

Message delivery guarentees
* At-least-once delivery => delivered at least one. Keep in mind if you have more than one instance of your application, each instance may get the message
* At-Most-Once-Delivery => each message IS NOT guaranteed for delivery. Very small chance it may never. But there is NO chance it will be delivered twice (Referred to automatic duplicate detection)
* First-in-First-Out => If you need messages to be processed in order, select this option

Transactional Support
* Group messages in a transaction (i.e. Shopping Cart, Charge Account, Shipping)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What to choose between Service Bus Topics vs Service Bus Queues

A

Topics
* Multiple receievers

Queues
* Need an At-Most-Once delivery
* FIFO
* Transactional Support
* Receive message w/o polling
** Need to handle messages larger thatn 64 kb but less than 100 mb. (standard tier message is limited to 256 kb)
Queue size will not exceed 1 TB*
* Want to publish and consume batches of messages or multiple destinations

Another part of the Microsoft documentation says 80 GB for queue max size

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Use Queue Storage if

A
  • Need an audi trail of all messages passing through queue
  • Expect queue to exceed 1 TB
  • Want to track progress for processing a message inside a queue
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Explain a good scenario for Event Grid

A

Someone upload a new song to Azure. A message is then sent to all subscribers subscribing to the Topic (i.e. Genre)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is Event Grid?

A

is a fully managed event routing service (running on top of the Azure Service Fabric).

It provides a dynamically scalable, low-cost messaging system that allow publishers to notify subscribers about a status change

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are the 5 components of Event Grid?

A
  • Events
  • Event Sources
  • Topics => the endpont where publishers send events
  • Event Subscriptions => The endpoint or built-in mechanism to route events. Subscriptions are also used by handlers to filter incoming events intelligently
  • Event Handlers
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is the max event/message size for Event Grid

A

64 kb

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What are some of the schema fields for an Event Grid message?

A
  • topic
  • subject
  • id
  • eventType
  • eventTime
  • data {}
  • dataVersion
  • metadataVersion
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What are System Topics?

A

They are built-in from Azure. You cannot see them in your subscription but you can subscribe to them.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What are custom topics?

A

Custom topics are application and third-party topics. When you create or are assigned access to a custom topic, you see that custom topic in your subscription.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is an event subscription?

A

Event Subscriptions define which events on a topic an event handler wants to receive. A subscription can also filter events by their type or subject, so you can ensure an event handler only receives relevant even

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What are some of the event sources for Event Grid?

A
  • Azure Subscriptions and Resource Groups
  • Container Registry
  • Event Hubs
  • Service Bus
  • Storage Accounts
  • Media Services
  • Azure IoT Hub
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are some of the Event Handlers for Event Grid?

A
  • Azure Functions
  • Azure Logic Apps
  • Webhooks
  • Event Hubs
  • Service Bus
  • Storage Queues
  • Microsoft Power Automate
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Why you should use Event Grid

A
  • Simplicity
  • Advanced Filtering => Subscriptions have close control over the vents they receive from a topic
  • Fan-out => you can subscribe to an unlimited number of endpoints to the same events and topics
  • Reliability => retrieves event delivery for up to 24 hours for each subscription
  • Pay-per-event => Pay for only the event that you transmit
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What are Event Hubs?

A

Similar to Event Grid, however it is optimized for extremely high thoughput, a large number of publishers, security, and resilency

Same publisher/subscriber pattern

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What are partitions in Event Hubs?

A

When an Event Hub receives communications, it divides them into partitions.
Subscribers can always use buffers to “catch up”. By default events stay in the buffer for 24 hours before expiring. The buffers are called partitions because the data is deviced amounst them. Each partition has a sperate set of subscribers.

Note, an event message can be set to a specific partition

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What is Capture in Event Hubs?

A

Event Hub can send all your events immediately to Data Lake or Azure Blob Storage for inexpensive, permanent persistence

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Authentication in Event Hubs

A

All publishers are authenticated and issued a token. This means Event Hubs can accept events from external devices and mobile apps, without worrying that fraudulent data from prankers could ruin our analysis.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Choose Event Hubs if

A
  • You need to support authenticating a large number of publisers
  • You need to save a stream of events to Data Lake or Azure Blob Storage
  • You need aggregation or analytics on your event system
  • You need reliabile messaging or resiliency

Can handle millions of events ver second with low latency. Great for real time analytics

27
Q

Choose Service Bus over Storage Queue if

A
  • You need an at-most-once delivery guarantee.
  • You need a FIFO guarantee (if no other settings preempt the default FIFO order)
  • You need to group messages into transactions.
  • You want to receive messages without polling the queue.
  • You need to provide role-based access to the queues.
  • You need to handle messages larger than 64 KB but smaller than 256 KB for the standard tier or 100 MB for the premium tier.
  • Your queue size won’t grow larger than 80 GB.
  • You’d like to be able to publish and consume batches of messages.

Choose Service bus too if you want to send with topics

28
Q

Choose Storage Queue over Service Bus if

A
  • You need a simple queue with no particular additional requirements.
  • You need an audit trail of all messages that pass through the queue.
  • You expect the queue to exceed 80 GB in size.
  • You want to track progress for processing a message inside the queue.
29
Q

Example of sending a message to a Service Bus Client

A

// Create a ServiceBusClient object using the connection string to the namespace.
await using var client = new ServiceBusClient(connectionString);

// Create a ServiceBusSender object by invoking the CreateSender method on the ServiceBusClient object, and specifying the queue name.
ServiceBusSender sender = client.CreateSender(queueName);

// Create a new message to send to the queue.
string messageContent = “Order new crankshaft for eBike.”;
var message = new ServiceBusMessage(messageContent);

// Send the message to the queue.
await sender.SendMessageAsync(message);

30
Q

To receive a message from Service Bus Client

A

// Create a ServiceBusProcessor for the queue.
await using ServiceBusProcessor processor = client.CreateProcessor(queueName, options);

// Specify handler methods for messages and errors.
processor.ProcessMessageAsync += MessageHandler;
processor.ProcessErrorAsync += ErrorHandler;

await args.CompleteMessageAsync(args.Message);

31
Q

Azure CLI to check messages in a queue

A

az servicebus queue show \
–resource-group [sandbox resource group name] \
–name salesmessages \
–query messageCount \
–namespace-name <namespace-name></namespace-name>

32
Q

What filters are available for Subscription Queue Topics?

A
  • Boolean (True means send all messages for the subscription and false means no messages are to be delivered for the subscription, basically disabling messages)
  • SQL Filters (Uses the SQL where clause and if true, delivers the messages)
  • Correlation (Holds a set of conditions that are matched again the properties of each message, if there is a match, the messages are delivered)
33
Q

Example of sending a topic

A

// Create the client
await using var client = new ServiceBusClient(connectionString);
// Create sender and specifiy the topic
ServiceBusSender sender = client.CreateSender(topicName);

string message = “Cancel! I have changed my mind!”;
var message = new ServiceBusMessage(message);
// Send the message to the topic.
await sender.SendMessageAsync(message);

34
Q

To receive a message from a subscription

A

processor = client.CreateProcessor(topicName, subscriptionName, options);
// Specify the handler method for messages.
processor.ProcessMessageAsync += MessageHandler;

// Specify the handler method for errors.
processor.ProcessErrorAsync += ErrorHandler;

// Complete the message. The message is deleted from the subscription.
await args.CompleteMessageAsync(args.Message);

35
Q

What is the maximum storage size for an Azure Storage Queue

A

500 TB

Need to verify this. I believe for Azure Service Bus is 80 GB and if you need more, they recommended Azure Queue Storage.

36
Q

How many messages per second can an Azure Storage Queue handle?

A

2000

37
Q

What is the maximum size for an mesage in Azure Service Bus

A

100 MB

38
Q

What is the maximum size for a message in Azure Storage Queue

A

64 KB

39
Q

What formats does Azure Storage Queue accept

A

JSON, XML, a simple string

40
Q

Are Azure Storage Queues available in what storage accounts?

A

V1 and V2

Blob Storage accounts only offer Blobs

41
Q

Some best practices for Azure Storage Queues

A
  • Should choose the same location as your source or destination components
  • Choose the approperiate replication model
  • Consider Access tiers (which only apply to blobs) and standard or premium accounts. Premium uses SSD
  • Require secure transfer if sensitive infomration may pass through a queue
42
Q

Azure CLI for creating a storage account

A

az storage account create
–name
-g
–kind (i.e. StorageV2
–sku (i.e. Standard_LRS)

43
Q

Authorization types in Azure Queues

A
  • Azure Active Directory
  • Shared Key (or account key)
  • SAS
44
Q

ACI to display a connection string for a storage account

A

az storage account show-connection-string
-g
-n
–output tsv

45
Q

Example of sending a Queue Message

A

QueueClient queueClient = new QueueClient(connectionString, queueName);

Response<SendReceipt> response = await queueClient.SendMessageAsync("This is a message");</SendReceipt>

string messageJson = JsonSerializer.Serialize(objectData);

Response<SendReceipt> response = await queueClient.SendMessageAsync(messageJson);</SendReceipt>

46
Q

Example on how to receive a message

A

Response<QueueMessage> response = await queueClient.ReceiveMessageAsync();</QueueMessage>

QueueMessage message = response.Value;

Console.WriteLine($”Message id : {message.MessageId}”);
Console.WriteLine($”Inserted on : {message.InsertedOn}”);
Console.WriteLine($”Message (raw) : {message.Body}”);

47
Q

Event Hubs

What is Azure Event Hubs

A

Azure Event Hubs is a cloud-based, event-processing service that can receive and process millions of events per second. Event Hubs acts as a front door for an event pipeline, to receive incoming data and stores this data until processing resources are available.

48
Q

Event Hubs

What is Azure Event Hubs

A

Azure Event Hubs is a cloud-based, event-processing service that can receive and process millions of events per second. Event Hubs acts as a front door for an event pipeline, to receive incoming data and stores this data until processing resources are available.

49
Q

What is the maximum size for a Event Hubs messages

A

1 MB

50
Q

What would be a scenario that is best for using AMQP (Advanced Message Queuing Protocol) with Azure Event Hubs

A

For publishers that send data frequently, AMPQ is better performance
(however higher initial session overhead because of a persistence bidirectinoaly socket and TLS or SSL/TLS has to be set up first)

51
Q

What would be a scenario that is best for using HTTP with Azure Event Hubs

A

More intermittent publishing, HTTPS is better
(requires more overhead with each request but there is no initial session overhead)

52
Q

Situation where to use Apache Kafka in Event Hubs

A

With endpoints compatible with Apache Kafka

53
Q

What are consumer groups?

A

An event hub consumer group represents a specific view of an event hub data stream

54
Q

What are the pricing tiers for Event Hubs?

A
  • Basic
  • Standard
  • Premium
  • Dedicated

Max Retention Period in Days: 1, 7, 90, 90
Storage Retention: 84 GB, 84 GB, 1 TB per PU, 10 TB, per CU
Can purchase more storage retention for Premium and Dedicated

55
Q

Azure CLI Commands to create an Event Hub

A

az eventhubs namespace create
–name

// Create the event hub
az eventhubs eventhub create
–name
–namespace

56
Q

Testing resilency in Event Hubs

A

Messages are still delivered to subscribers even after the Event Hub goes down. You can test this by temporarly disabling the Event Hub in the portal, then re-enabled. The receiving app should receive messages again

You can then check the following metric
* Throttled Requests => The number of throttled request because the throughput exceeded unit usage
* Active Connections
* Incoming/Outgoing

57
Q

What is an Event Grid?

A

Event Grid aggregates all your events and provides routing from any source to any destination. Event Grid is a service that manages event routing and delivery from many sources and subscribers. This process eliminates the need for polling and results in minimized cost and latency.

58
Q

What is the capabilities of Event Grids?

A
  • It’s simple
  • It can filter events
  • It supports multiple subscribers
  • It’s reliabile
  • Its throughput is high
  • It has built-in events
  • Is supports custom events
59
Q

What are some of the Event Handlers for Event Grid?

A
  • Azure Functions
  • Azure Logic Apps
  • Azure Automation
  • Azure Event Hubs
  • Azure Service Bus
60
Q

By default, how many partitnios will a new Event Hub have?

A

4

61
Q

What is the maximum size for a single publication (individual or batch) allowed by Azure Event Hubs?

A

256 KB

62
Q

Diagram for Azure Event Hubs

A
63
Q

High level overview of Event Hubs, Event Grids, and Azure Service Bus

A
64
Q

Using the Event Hub and Event Grid together

A