Database Specialty - Amazon QLDB

Question 1

Amazon QLDB – Overview

Answer 1

QLDB = Quantum Ledger Database
* Fully managed, serverless ledger database
* Has built-in, immutable journal to record all the change history of your data
* Transparent and cryptographically verifiable ledger
* Tracks each application data change and maintains a complete and verifiable history of
changes over time
* Supports ACID transactions
* Uses query language named PartiQL (SQL-like, Open standard)

• Uses Amazon ION format
  
  • A Superset of JSON
  • Self-describing, hierarchical data serialization format
  • Offers interchangeable binary and text representations
  • Adds additional data types, type annotations and comments to JSON format
  • Supports nested JSON elements
  • Use cases: System of record applications like Banking transactions, HR services records, Insurance claim histories, Vehicle ownership records etc.

Question 2

QLDB Architecture

Answer 2

• It’s serverless (scales automatically to support the needs of your application)
• Intended to support high-performance OLTP workloads

Question 3

QLDB Architecture

Answer 3

• Ledger = Journal + set of tables
• Journal
  • Is append-only, immutable
  • No updates / overwrites / deletes
  • Stores a sequenced, cryptographically verifiable entry of each change to the
    table data
  • Changes are chained together as blocks (but not a blockchain implementation)
  • QLDB is centralized and not a distributed ledger (blockchain is used with
    decentralized use-cases)
  • Even if you delete data from the ledger (table), you can access its change history
    from the immutable journal

Question 4

QLDB Architecture

Answer 4

• Tables
  
  • Collection of documents and their revisions
  • Store the current and historical state of your data (indexed storage)
  • Can include document deletion records
  • Documents are in ION format

Question 5

QLDB Views

Answer 5

• QLDB offers three views of your data
  
  • User view
  • Committed view
  • History view
• User view
  
  • latest version of your data
  • default view
• Committed view
  
  • user view + system generated metadata
• History view
  
  • contains all historical document revisions
  • i.e. all change history with metadata

Question 6

Working with QLDB

Answer 6

• You create a ledger and define your tables
• QLDB supports ACID semantics
• We use PartiQL query language to query QLDB
  
  • It’s a SQL-like open standard query language
  • SQL-compatible access to relational, semi-structured, and nested data
  • Extends SQL to support ION documents
  • PartiQL is also used with Redshift / S3 Select / Glacier Select

Question 7

Amazon ION format

Answer 7

• Is a Superset of JSON
• Self-describing, hierarchical data serialization format (=nested JSON)
• Offers interchangeable binary and text representations
• Adds additional data types, type annotations and comments to JSON
  format
• Flexible data model

Question 8

Data Verification in QLDB

Answer 8

• Journal maintains immutable and verifiable
  transaction log
• QLDB uses a digest for verification
• Digest
  
  • is a cryptographic representation of your journal * or a unique signature of your data’s entire change history as of a point in time
  • is generated using SHA-256 hash function with a Merkle tree
    -based model
• Can verify the integrity of your data by calculating the digest and comparing it with QLDB’s digest
• Can verify using the AWS console or QLDB API
• Improper verification requests typically result in IllegalArgumentException

Question 9

QLDB Backup and Restore

Answer 9

• QLDB does not support a backup and restore feature (yet!)
• PITR is also not supported (yet!)
• Can only export your QLDB journal to S3
  
  • For analytics/auditing / data retention / verification / exporting to other systems
  • limit of two concurrent journal export jobs

Question 10

QLDB Streams

Answer 10

• Continuous flow of data from your ledger’s journal to a Kinesis data stream
• Provides an at-least-once delivery guarantee
• No ordering guarantees
  
  • Revisions can be produced in a Kinesis data stream out of order

Question 11

QLDB High Availability and Durability

Answer 11

• QLDB ledger is replicated across multiple AZs within the region (=high availability)
• With multiple copies per AZ (=strong durability)
• Write is acknowledged only after being written to a durable storage in multiple
  AZs
• CRR is not supported (yet!)
  
  • QLDB journal can be exported to an S3 bucket
  • S3 bucket can then be configured for CRR

Question 12

QLDB Security

Answer 12

• IAM is used for authentication and authorization of QLDB resources
• Supports encryption at rest and in transit
• Uses Amazon-owned keys to encrypt QLDB data
• Does not support CMKs

Question 13

QLDB Security - Networking

Answer 13

• Can use an interface VPC endpoint to allow VPC resources to connect to
  QLDB privately
• Interface VPC endpoints are powered by AWS PrivateLink
• PrivateLink provides private and secured connectivity between VPCs,
  AWS services, and on-premises applications
• PrivateLink eliminates the need for IG / NAT device / VPN connection / or
  AWS Direct Connect connection

Question 14

QLDB Monitoring

Answer 14

• Integrated with CloudWatch (Alarms / Logs / Events)
• Common metrics
  
  • JournalStorage
  • IndexedStorage
  • ReadIOs
  • WriteIOs
  • CommandLatency
• QLDB log files provide additional information
• API calls and user activity can be logged with CloudTrail

Question 15

QLDB Pricing

Answer 15

• You pay only for what you use
• Storage – per GB per month
  
  • Journal Storage and Indexed Storage
• IOs – per million requests
  
  • read IOs and write IOs
• Data transfer