GCP & ML Fundamentals

Question 1

Big data challenges are…

Answer 1

• Migrating existing data solutions
• Analysing large datasets at scale
• Building streaming pipeline
• Applying machine learning

Question 2

Google Cloud platform is made up of three layers…

Answer 2

Top: Big Data & ML Products

Middle: Compute Power, Storage, Networking

Bottom: Security

Question 3

Google cloud bucket names have to be…

Answer 3

Globally unique.

Question 4

What is the command line command for google cloud?

Answer 4

gsutil

Question 5

Are we charged when a virtual machine is ‘stopped’?

Answer 5

Yes - but only for the disk space (keep the VM and the software which is installed on it).

Question 6

Why does cloud computing differ from desktop computing?

Answer 6

In cloud computing - compute power and storage are independent.

Question 7

What are the 4 types of google cloud storage?

Answer 7

Standard, Nearline, Coldline and Archive

Question 8

What is Stardard Storage?

Answer 8

Best for frequently accessed data and for data which is not going to be stored for a long time.

Question 9

What is nearline storage?

Answer 9

Low cost and highly durable for data which is read/modified once a month.

Question 10

What is coldline storage?

Answer 10

Lower cost than nearline and highly durable for data which is read/modified once a quarter.

Question 11

What is archive storage?

Answer 11

Lowest cost for storing data, archived as a online backup.

Question 12

What is a project?

Answer 12

Base level organising entity for creating and using resources, managing billing, APIs and permissions.

Question 13

What is a folder?

Answer 13

A folder contains multiple projects within an organisation.

Question 14

What is the root node of the GCP hierarchy?

Answer 14

Organisation

Organisation -> Folder -> Project -> Resources e.g. BQ dataset

Question 15

What google service controls GCP resources?

Answer 15

Identity and Access Management

Question 16

What is IAMs and what does it do?

Answer 16

Identity and Access Management - it allows you to fine tune access control to all the GCP resources in use.

Question 17

What is the main benefit of networking?

Answer 17

We don’t have to do everything on one machine if we have a fast enough network. Googles data center network enables separation of compute and storage (process data without copying it).

Question 18

What are edge points of presence (in networking)?

Answer 18

Edge points of presence are where the private google network connects to the public one.

Question 19

What are the different levels of security you can have?

Answer 19

• On-premise (full responsibility)
• Infrastructure as a Service (IaaS)
• Platform as a Service (PaaS)
• Managed Services (Least Responsibility)

Question 20

What google tool helps to implement security policies?

Answer 20

IAMs (Identity and Access Management)

Question 21

How can BigQuery be accessed?

Answer 21

Command Line, REST API, Web UI and third party tools (e.g. matillion).

Question 22

What is Compute Engine?

Answer 22

Lets you run virtual machines on demand in the cloud = IaaS solution.

Question 23

What are the 3 types of compute engine available on GCP?

Answer 23

1. Custom Machines - optimal cpu and memory for problem.
2. Spot Machines - reduce computing cost by 91%
3. Predefined Machines

Question 24

What is Google Kubernetes Engine (GKE)?

Answer 24

GKE is clusters of machines running containers, it is way to orchestrate code which is running in containers.

Question 25

What is a container?

Answer 25

A container is a package of code and its dependencies - it is highly portable and resource efficient.

Question 26

What is App Engine?

Answer 26

Googles PaaS which is a way to run code without worrying about infrastructure.

Question 27

What are Cloud Functions?

Answer 27

Serverless execution environments (FaaS (function as a service)) - it executes code in response to an event and google scales resources as required.

Question 28

What are Googles database managed services?

Answer 28

Cloud BigTable, Storage, SQL, Spanner and Datasource.

Question 29

What roles exist in an analytics team?

Answer 29

• Data Engineer
• Decision Makers
• Analysts
• Statisticians
• Applied ML Engineer
• Data Scientists

Question 30

What is a recommendation system?

Answer 30

Model recommends products based on preferences e.g. Netflix or YouTube.

• They must scale to meet demand.
• Prediction can be stream or batch.

Question 31

Where should you store unstructured data?

Answer 31

Cloud Storage

Question 32

Where should you store data which is structured and has a latency in seconds?

Answer 32

BigQuery

Question 33

Where should you store data which is structured and has a latency in milli-seconds?

Answer 33

Cloud BigTable

Question 34

Where should you store data which is structured and is a No SQL workload?

Answer 34

Cloud DataStore

Question 35

Where should you store data which is structured, is SQL based and where 1 database is enough?

Answer 35

Cloud SQL

Question 36

Where did Big Data Tools evolve from?

Answer 36

• Hadoop = MapReduce (map; performs filter and reduce; summary operations).
• Cloud Services = seperate, specialise and connect.

Question 37

What are clusters?

Answer 37

Clusters are a fungible resource, they are used when required and manged automatically through dataproc.
- Clusters can be up and running in 90seconds.

Question 38

What can we do to clusters to avoid under or over provisioning?

Answer 38

• Auto Scaling = turn on or off according to job size.

- Incorporate Premptible VMs - affordable (80% cheaper) and short lived but limited.