Storing Data Flashcards
Identity and briefly explain the 3 different data types
Structured Data
* Relational containings sames fields and properties
* Updates can be difficult due to constriaints
Semi-Structured
* Non-relational
* More key values
* NoSQL and data serialization into XML, JSON, YAML
Unstructured Data
* Media Files
* Microsoft 365 files
* Text Files
* Log files
Determine Operational Needs, questions to consider
- Doing simple lookups?
- Query for one or more fields?
- Number of create, update and delete operations
- Queries run complex analytical operations?
- How quickly do these operations need to be processed?
Some examples include Product Catelog, Photos and Videos, and Business Data (All 3 needing have different operational needs)
Identity and briefly explain ACID prinicpals?
- Atomicity => means a transaction must execute once and it must be atomic (all of it completed or none at all)
- Consistency => data is consistent before and after the transaction
- Isolation => ensures each transaction is not affected by other transactions
- Durability => changes made as a result of the trasnaction are permantly saved in the system
What is OLTP?
Online Transaction Processing
High usages and volumes of data, short transactions
What is OLAP?
Online Analytics Processing
Less users, longer response times, large volume of data
Regarding the Product Catalog, Identity the following:
* Data Classification
* Operations
* Latency and Throughput
* Transaction Support
* Recommendation
NOTE: Could have used Azure SQL, however products may not have similar properties
Regarding storage and transaction needs of Photos and Videos, Identity the following:
* Data Classification
* Operations
* Latency and Throughput
* Transaction Support
* Recommendation
Regarding storage and transaction needs of Business Data, what would be the recommendation?
Can use Azure SQL Database but could also consider Azure Analysis Services
What are the considerations when determining the number of storage accounts?
Data Diverisity
* Data specific to country or region?
Costs
* Geo-redunancy more that local redundant
* Premium vs Standard Tier
* Hot access increases costs (access tier)
* You could split storage accounts and put critical data in geo-redundant and less critical use local redundant
Management Overhead
* Requires time to create and maintain
Three things to consider when choosing Account Settings
- Name => 3 - 24 characters and must be unique
- Deployment Model => Resource Manager (recommended) or Classic (Azure Service Manager API)
- Account Kind => Storage V2, Storage V1, and Blob Storage
Identity the various Account Creation Tools
- Azure Portal
- Azure CLI
- Azure PowerShell
- Management Client Libraries
Identity the 4 terms that define Microsoft Azure Storage
- Managed
- Durable => redundant
- Secure
- Scalable
How many Storage Accounts can a subscription have PER REGION
250
What are the 4 types of data supported by Azure Storage
- Blobs
- Fileshares
- Queues
- Table Storage
What are the 3 Blob Types?
- Block Blob
- Page Blob
- Append Blob
What are the features of the File Share?
- Enables you to set up highly available network file shares that can be accesed the the Server Message Block (SMB) Protocol
- REST interface
- Storage Client Libraries
- Uses (Configuration Files, Log Files)
Az Command to create a Storage Account
az storage account create
–resource-group
–location
–sku
–name
HTML to list all blobs in a container
GET https://[mystorageaccount]/?comp=list&include=metadata
Shows results in XML
Basic Example using the client libraries
string containerName = “mycontainer”;
BlobContainerClient container = new BlobContainerClient(connectionString, containerName);
var blobs = container.GetBlobs();
foreach (var blob in blobs)
Console.WriteLine($”{blob.Name}”);
Basic API Endpoints
- https://{name}.blobs.core.windows.net
- https://{name}.queue.core.windows.net
- https://{name}.table.core.windows.net
- https://{name}.file.core.windows.net
Whats the difference between Access Keys and SAS
Access Keys
* Similar to user name and password
* Rotate Keys
* Shouldn’t put in configuration files or hard coded in application
** SAS**
* Limited permissions to a particular resource
Az Command to show your Storage Account connection String
az storage account show-connection-string
–resource-group
–query connectionString
–name
Shows results in XML
Storage Account Security Considerations
- Protect data at rest (this is automatic and cannot be disabled)
- Protected data in transit (use HTTPS)
- Support browser cross-domain access
- Use RBAC
- Audit storage files (logs)
Embed the SAS in an HTML header
Authorization: SharedKey myaccount:cY49k44BimLToTWiloX44ldfas0pi833Ppxda
What two type of SAS are provided?
- Service Level SAS
- Account Level SAS
For untrusted clients, use SAS
Two typical designed to get SAS keys
- Front End Proxy Service
- SAS Provider Service
What are the options when selecting network access?
- Enable from all networks
- Enable from selected networks and IP Address
- Disable
What are the features of Microsoft Defender?
- Provides an extra layer of security
- Available for blob and files General Purpose V2
- Not available to foreign governments
- Send alerts to administrators
What are some of the details in an event notification from Defender?
- Nature of anomaly
- Storage Account Name
- Event Time
- Storage Type
- Protection causes
- Investigation Steps
- Remediation stps
- Also includes details about possible causes and recommended actions to investigate and mitigate the potential threat
Azure Data Lake Storage
- Provides first-class data lake solution than enables enterprises to consolidate their data (its built on Azure Blob Storage)
- RBAC, Access Control List
- Authenticates through AD OAuth 2.0 bearer tokens and can include MFA
- End to End encryptions
