AWS Snowball Edge | Clustering

Question 1

Can I access files migrated via the file interface on Snowball Edge from Amazon EFS?

Clustering

AWS Snowball Edge | Storage

Answer 1

No. Files migrated via the file interface on Snowball Edge can be accessed directly as objects in S3 or as files through the AWS Storage Gateway.

General Information

Question 2

What is the clustering feature in Snowball Edge?

Clustering

AWS Snowball Edge | Storage

Answer 2

The clustering feature in Snowball Edge allows you to aggregate many Snowball Edge devices together to create one logical compute and storage pool with increased durability and capacity.

Question 3

Why should I use the clustering feature in Snowball Edge?

Clustering

AWS Snowball Edge | Storage

Answer 3

Clustering Snowball Edge devices together creates a more durable, scalable, local storage pool. Snowball Edge clusters allow you to access large amounts of local storage capacity by using multiple Snowball Edge devices.

Question 4

Can I use Snowball Edge in a clustered configuration to augment or replace my data storage solution on premises?

Clustering

AWS Snowball Edge | Storage

Answer 4

Yes. The scalable storage that comes with Snowball Edge is designed to store your data safely and durably.

Question 5

What are some scenarios for using the Snowball Edge clustering feature?

Clustering

AWS Snowball Edge | Storage

Answer 5

You can use the clustering feature in Snowball Edge when you have a need to use durable storage on-premises. Some customers occasionally face situations where they need durable storage in remote sites where it is difficult to have large amounts of data storage. For example, in manufacturing there are companies who want to have a pool of storage for every factory location; or in the military there are divisions that have shifting data storage requirements and need the flexibility to increase and decrease storage capacity for their vehicles.

Creating AWS Snowball Edge Clusters

Question 6

How do I get started using a Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 6

You can order a Snowball Edge cluster using the AWS Snowball API, AWS Console, AWS SDK or AWS CLI. Once you create a cluster, you will receive a cluster ID (beginning with CID) which you will use to refer to any operations on this particular cluster job.

Question 7

Is there a minimum size for my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 7

Yes, the minimum size of a Snowball Edge cluster is 5 devices, or nodes.

Question 8

Is there a maximum size for my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 8

Yes, the maximum size of a Snowball Edge cluster is 20 devices, or nodes.

Question 9

What is the usable space in a Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 9

The total size of a Snowball Edge cluster is based on the number of nodes and usable capacity per node, which is 45TB. For example, in a 5-device cluster with 500TB of physical capacity, you will effectively realize usable space of 225TB. After configuring your client, you can get the total used space and total available space in your cluster from the command line.

Question 10

Can I build a cluster from individually provisioned Snowball Edge devices?

Clustering

AWS Snowball Edge | Storage

Answer 10

No. The only way to start using a cluster is to create a job of the type “AWS Snowball Edge cluster”.

Starting Up an AWS Snowball Edge Cluster

Question 11

Once the Snowball Edge devices arrive at my location, how do I setup the cluster?

Clustering

AWS Snowball Edge | Storage

Answer 11

Once the Snowball Edge devices have all arrived at your location, you need to power them on, connect them to the same network, download the manifest file and unlock code, download the Snowball client, and “unlock” the cluster.

Question 12

Can I issue management commands and reads/writes to any node in a cluster?

Clustering

AWS Snowball Edge | Storage

Answer 12

Yes. Each Snowball Edge node in the cluster is capable of both reads and writes using S3 commands. You can also mount each node separately and perform simultaneous reads and writes across the cluster. Snowball Edge commands can be used across any node of a cluster to perform management functions like unlock, describe, etc.

Question 13

How do I unlock my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 13

Before starting the unlock process, please ensure that you have at least N-2 nodes of your cluster available to you, where N is the maximum number of nodes in the cluster. Note the IP addresses of all the Snowball Edge devices from the LCD screen. To unlock the Snowball Edge cluster you will connect to any one of the nodes using the client software, configure your client, and type the unlock command.

Question 14

Do I need to unlock each node of my Snowball Edge cluster separately?

Clustering

AWS Snowball Edge | Storage

Answer 14

No. You can unlock all nodes in a cluster by issuing the “unlock-cluster” command to a single node in your cluster.

Question 15

Can I unlock certain nodes of my Snowball Edge cluster before I receive all devices or do I need to wait until I have all devices in my possession?

Clustering

AWS Snowball Edge | Storage

Answer 15

We do not recommend you start your Snowball Edge cluster with less than the number of nodes that the cluster job is provisioned with since you may, or may not, be able to read and/or write durably to the cluster depending on the number of nodes.

Question 16

How do I know what the total number of nodes are in my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 16

After configuring your client, you can run a simple command to learn more about the number of nodes that are available in your Snowball Edge cluster.

Question 17

What are the different types of status that a particular device in a Snowball Edge cluster can be in?

Clustering

AWS Snowball Edge | Storage

Answer 17

A device can report a few different statuses. Here is a brief description of the different types of status that a Snowball Edge can report:

Unlock Status: Indicates whether a node is unlocked.

Reachability Status: Indicates whether a device is reachable on the network.

Cluster Association Status: Indicates whether a device is associated with a cluster.

Extending Nodes in an AWS Snowball Edge Cluster

Question 18

Is my Snowball Edge cluster expandable in size by adding nodes?

Clustering

AWS Snowball Edge | Storage

Answer 18

You can add a new node to your Snowball Edge cluster if you have removed an unhealthy node from the cluster, or if you are adding another node for increased local storage. To add a new node, you first need to order a replacement. Ordering a replacement node can be done from the console, from the AWS CLI, or from one of the AWS SDKs.

Replacing Nodes in an AWS Snowball Edge Cluster

Question 19

Can I remove a node from my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 19

You can remove a node from your Snowball Edge cluster if you intend to replace it. You may want to do this if one of your nodes is unhealthy, is not powering on, or is defective. To remove a node, you can power it off and use a simple disassociate command.

Question 20

How do I replace a node in my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 20

To order a replacement node in your Snowball Edge cluster, you can go to the AWS Console page, and order a replacement device. Once you receive the replacement device, you need to follow a few steps:

Remove the unhealthy device from the cluster by powering it off and use the “disassociate-device” command.

Plug-in the replacement Snowball Edge device and wait about 10 minutes for it to boot. If DHCP is being used then the device will obtain an IP address automatically. If you are using a static IP address then you will need to manually assign an IP address.

Add the replacement Snowball Edge device and “associate” it.

Question 21

What happens if I remove more than one Snowball Edge device from my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 21

You cannot remove more than one Snowball Edge device from a Snowball Edge cluster at any time.

Question 22

Can I continue to use the Snowball Edge cluster while I am replacing a Snowball Edge device?

Clustering

AWS Snowball Edge | Storage

Answer 22

Yes. You can continue to use the Snowball Edge cluster even when you are replacing a Snowball Edge device. Until all the nodes are ASSOCIATED and REACHABLE, the cluster will run in a reduced durability mode.

Question 23

What will happen to stored data once I replace a node from my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 23

Once you replace a node, the Snowball Edge cluster will repartition the data amongst the remaining nodes so that it can still be used as durable storage.

Question 24

Can I choose different job settings (Address, Shipping speed, Lambda functions, IAM roles, KMS keys, SNS notifications) in my replacement node than what was set when I ordered the Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 24

No. Each replacement node is pre-configured to use the same job configurations that came with the Snowball Edge cluster.

Question 25

Can I choose a different Lambda function in my replacement node?

Clustering

AWS Snowball Edge | Storage

Answer 25

Yes. You can choose a different Lambda function as long as it uses the same S3 bucket that was originally configured with the Snowball Edge cluster.

Question 26

Can I use the same Snowball Edge cluster manifest key and unlock code as before after ordering a replacement node?

Clustering

AWS Snowball Edge | Storage

Answer 26

No. A new cluster manifest and unlock code is generated after your order of a replacement device which must be used with any future operations of the Snowball Edge cluster.

AWS Snowball Edge Cluster Management

Question 27

What happens if one of my nodes is not available?

Clustering

AWS Snowball Edge | Storage

Answer 27

You can continue to use the Snowball Edge cluster even if it runs in a reduced durability level. Immediately order a replacement Snowball Edge device so that you can replace this node with a newer device.

Question 28

What happens if two of my nodes are not available?

Clustering

AWS Snowball Edge | Storage

Answer 28

You can continue to use the Snowball Edge cluster but note that it will be running in a read-only mode and existing data on the cluster is at risk. Immediately order two replacement devices so that you can replace these nodes with newer Snowball Edge devices.

Question 29

What happens if three or more of my nodes are not available?

Clustering

AWS Snowball Edge | Storage

Answer 29

You cannot use the Snowball Edge cluster anymore and existing data on the cluster is at risk. Contact AWS support immediately if three or more Snowball Edge devices are non-functional.

Question 30

How long can I keep a Snowball Edge cluster on premise at my location?

Clustering

AWS Snowball Edge | Storage

Answer 30

The maximum duration a Snowball Edge cluster can be kept on premise at your location is 360 days from the day the order was created. After that, the certificates in the cluster expires and either all the Snowball Edge devices in the cluster must be returned or you should order new devices.

Question 31

Can I pre-order replacement devices for my Snowball Edge cluster?

Clustering

AWS Snowball Edge | Storage

Answer 31

Yes. To pre-order replacement devices, you need to create a Snowball Edge cluster job and then create a job to order replacement Snowball Edge devices. The replacement devices do not need to be connected to the original Snowball Edge cluster until one of the devices is unhealthy. Once a Snowball Edge device is unhealthy, you can remove the device and then add the replacement device.

Ingesting Data with AWS Snowball Edge Clustering

Question 32

Can a Snowball Edge cluster come preloaded with data from an existing S3 bucket?

Clustering

AWS Snowball Edge | Storage

Answer 32

No. However, you can place an order for two Snowball jobs – one cluster job and one export job. The Snowball Edge cluster will come empty from AWS but the export job can be a Snowball (or a Snowball Edge) device that can be used to import data into the Snowball Edge cluster.

Question 33

Can I ingest data from the entire Snowball Edge cluster back to AWS using individual nodes of my cluster?

Clustering

AWS Snowball Edge | Storage

Answer 33

No. A Snowball Edge cluster can only be used for on-premise storage and compute. It cannot be used to import data into AWS. To import data from your cluster, you will need to order additional AWS Snowball (or AWS Snowball Edge) devices and then connect them to the same network as the Snowball Edge cluster and transfer your data from the cluster. Depending on the total amount of data used in the cluster, you may have to order multiple import devices to ship the data back to AWS.

Question 34

When writing data into a cluster, does the network throughput of data transfer increase depending on the number of nodes?

Clustering

AWS Snowball Edge | Storage

Answer 34

Yes. The read and write throughput can be higher if you read and write from several devices in parallel.

Using Compute with AWS Snowball Edge Clustering

Question 35

Does compute capacity scale as I scale my cluster?

Clustering

AWS Snowball Edge | Storage

Answer 35

Yes. With more devices, you increase the compute capacity of your cluster.