Cloud Dataproc Flashcards

1
Q

What is Cloud Dataproc?

A

A fully managed cluster data processing service (Apache Spark and Apache Hadoop service)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are key points of Cloud Dataproc?

A
  1. Compatible with Apache Hadoop, Spark and Hive
  2. Runs in clusters
  3. Allows existing projects to be moved without redevelopment
  4. Fast cluster creation - can create workflow templates
  5. Can scale clusters without stopping Job
  6. Can switch to different versions
  7. Can handle streaming and batch data
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

When do you choose Cloud Dataproc over Cloud Dataflow?

A

If you have dependencies on Hadoop or Spark, or if you want more hands on management and control.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

How do you create a Cloud Dataproc cluster from the command line?

A

gcloud dataproc clusters create [CLUSTER NAME] –zone [ZONE]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

How do you submit a job to Cloud Dataproc via the shell?

A

gcloud dataproc jobs submit [TYPE] –cluster [CLUSTER NAME] –jar [JAR FILE]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What cluster modes can you choose when setting up Cloud Dataproc?

A
  1. Single - for development
  2. Standard - one master node
  3. High Availability - uses 3 master nodes
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What job types are available for Cloud Dataproc?

A
  1. Spark
  2. PySparck
  3. SparkR
  4. Hive
  5. Spark SQL
  6. Pig
  7. Hadoop
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

How do you import or export data to Cloud Dataproc?

A

You don’t. It’s a data analysis platform, not a database.
You can import and export to save/restore the cluster configuration data.
gcloud beta dataproc clusters export [CLUSTER NAME] – destination=[PATH TO EXPORT FILE]
gcloud beta dataproc clusters import [SOURCE FILE]

How well did you know this?
1
Not at all
2
3
4
5
Perfectly