FSx for Lustre

Question 1

FSx for Lustre

Answer 1

Managed Lustre - Designed for HPC - LINUX Clients

Supports Posix style permission (POSIX)

Single AZ Only

Question 2

FSx for Lustre Performance

Answer 2

100’s GB/s throughput & sub millisecond latency
- Available over VPN and DX

Question 3

Deployment types

Answer 3

1. Persistent
2. Scratch

• Scratch - Highly optimised for Short term no replication & fast
• Persistent - longer term, HA (in one AZ), self-healing

AWS
*Scratch file systems are designed for temporary storage and shorter-term processing of data. Data is not replicated and does not persist if a file server fails.

Persistent file systems are designed for longer-term storage and workloads. The file servers are highly available, and data is automatically replicated within the AWS Availability Zone (AZ) that is associated with the file system. The data volumes attached to the file servers are replicated independently from the file servers to which they are attached.*

Question 4

How Lurstre stores data

Answer 4

The secret behindLustre’s High performance is its storage, caching, and Networking Architecture:

• Lustre Metadata are stored on Metadata Targets (MST)
• Objects are stored on called object storage targets (OSTs) (1.17TiB)
• All Connections to Lustre is via a single client ENI in my vpc.
• Baseline performance based on size
• Size - min 1.2TiB then increments of 2.4TiB
• For Scratch - Base 200 MB/s per TiB of storage
• **Persistent* offers 50MB/s , 100MB/s and 200 MB/s per TiB of storage
• Both can Burst up to 1,300 MB/s per TiB (Credit System)

Question 5

Scatch Deployment type

Answer 5

Scratch is designed for pure performance
FSX sits in an AZ on its own and serves the vpc through the ENI

• Short term or temp workloads
• NO HA.. NO REPLICATION
• Larger file systems means more servers, more disks and more chance of failure !!
• Backup (Automatic and Manual

Question 6

Persistent Deployment

Answer 6

• Persistent deployment has its replication within ONE AZ only
• Auto-heals when hardware failure occurs(Hardware and not AZ Failure
• You can backup to S3 with both Manual or Automatic (0-35 day retention)

Question 7

How can I set and enforce storage limits for file system users?

Answer 7

You can set and enforce storage limits based on the number of files or storage capacity consumed by a specific user, group or project. You can choose to set a hard limit that denies users, groups or projects from consuming additional storage after exceeding their quota, or set a soft limit that provides users with a grace period to complete their workloads before converting into a hard limit. To simplify file system administration, you can also monitor user-, group- and project-level storage usage on FSx for Lustre file systems. To learn more, visit the FSx for Lustre Storage Quotas documentation.

Question 8

How do I see the impact of data compression on my file system storage?

Answer 8

You can use CloudWatch Metrics to see the total logical disk usage (without compression) and total physical disk usage (with compression) of your file system. See Amazon FSx for Lustre Data Compression documentation for additional information.

Question 9

NFS vs SMB

Answer 9

NFS and SMB are two widely used network file sharing protocols that have been developed for different operating systems and environments. NFS is known for its fast performance and low overhead, while SMB is known for its reliability and compatibility