General

Question 1

Syntax for Generated Column

Answer 1

GENERATED ALWAYS AS (CAST(orderTime as DATE))

Question 2

What is the main difference between AUTO LOADER and COPY INTO?

Answer 2

Auto loader supports both directory listing and file notification but COPY INTO only supports directory listing.

Question 3

Why does AUTO LOADER require schema location?

Answer 3

Schema location is used to store schema inferred by AUTO LOADER

Explanation: the next time AUTO LOADER runs faster since it does not need to infer the schema every single time by trying to use the last known schema.

Question 4

You are designing a data model that works for both machine learning using images and Batch ETL/ELT workloads. Which of the following features of data lakehouse can help you meet the needs of both workloads?

Answer 4

Data lakehouse can store unstructured data and support ACID transactions

Question 5

Where does Databricks architecture host jobs/pipelines and queries?

Answer 5

Control Plane

Question 6

Databricks Repos can implement what CI/CD operations?

Answer 6

Pull the latest version of code into production folder

Explanation: Not stuff like PRs and reviews - that’s handled by github (e.g.)

Question 7

What’s the syntax for creating or overwriting an existing delta table?

Answer 7

CREATE OR REPLACE TABLE

Explanation: When creating a table in Databricks by default the table is stored in DELTA format

Question 8

When a managed table is dropped, what happens to the data, metadata, and history?

Answer 8

They are also dropped from storage

Question 9

When a notebook is detached and re-attached, what happens to session-scoped temporary views?

Answer 9

They are lost

Question 10

When a notebook is detached and re-attached, what happens to global temporary views?

Answer 10

They can still be accessed

Question 11

Use colon (:) syntax in queries to access subfields in ____ and use period (.) syntax in queries to access subfields in ____

Answer 11

JSON strings

Struct types

Question 12

Assert syntax

Answer 12

assert row_count == 10, “Error message”

Question 13

Python error handling

Answer 13

try: except:

Question 14

Python Spark Syntax to create a view on top of the delta stream(stream on delta table)?

Answer 14

Spark.readStream.table(“sales”).createOrReplaceTempView(“streaming_vw”)

Question 15

You are currently asked to work on building a data pipeline, you have noticed that you are currently working with a data source that has a lot of data quality issues and you need to monitor data quality and enforce it as part of the data ingestion process, which of the following tools can be used to address this problem?

Answer 15

Delta Live Tables

Explanation: Delta live tables expectations can be used to identify and quarantine bad data, all of the data quality metrics are stored in the event logs which can be used to later analyze and monitor.

Question 16

What are the different ways you can schedule a job in Databricks workspace?

Answer 16

Immediate, CRON, Continuous, when new files arrive

Question 17

Databricks SQL queries are running slow. All the queries are running in parallel and using a SQL endpoint(SQL Warehouse) with a single cluster. What can you do to improve the performance/response times of the queries?

Answer 17

Increase the maximum bound of the SQL endpoint’s scaling range.

Explanation: The question is looking to test your ability to know how to scale a SQL Endpoint(SQL Warehouse) and you have to look for cue words or need to understand if the queries are running sequentially or concurrently. if the queries are running sequentially then scale up(Size of the cluster from 2X-Small to 4X-Large) if the queries are running concurrently or with more users then scale out(add more clusters).

Question 18

What does the Auto Stop feature do?

Answer 18

It automatically terminates the cluster when you are not using it

Question 19

Unity catalog simplifies managing multiple workspaces, by storing and managing permissions and ACL at _______ level

Answer 19

Account

Question 20

What section in the UI can be used to manage permissions and grants to tables?

Answer 20

Data Explorer

Question 21

What is not a privilege in the Unity catalog?

Answer 21

DELETE

Explanation: DELETE and UPDATE permissions do not exit, you have to use MODIFY which provides both Update and Delete permissions.

Also: TABLE ACL privilege types are different from Unity Catalog privilege types, please read the question carefully.

Question 22

Syntax for transferring ownership of a table to a group

Answer 22

ALTER TABLE table_name OWNER to ‘group’

Question 23

What is the array function that takes an input column and returns a unique list of values in an array?

Answer 23

collect_set()

Question 24

What is the default location where spark stores user databases?

Answer 24

dbfs:/user/hive/warehouse

Question 25

When can INSERT OVERWRITE update the schema?

Answer 25

when spark.databricks.delta.schema.autoMerge.enabled is set true

Question 26

Which of these is NOT a valid messaging option for job notifications? (SMS, Email, PagerDuty, Messaging Webhook, SES, SNS)

Answer 26

SMS

Question 27

Databricks Web Application is hosted in the Control Plane or Data Plane?

Answer 27

Control Plane

Question 28

Notebooks and Jobs are hosted in the Control Plane or Data Plane?

Answer 28

Control Plane

Question 29

What are the output modes for the trigger command while writing to a streaming table?

Answer 29

Append and Complete

Question 30

What command can be used to write data into a Delta table while avoiding the writing of duplicate records?

Answer 30

MERGE

Question 31

Why does AUTO LOADER require schema location?

Answer 31

Schema Location is used to store the location of schema inferred by the AutoLoader

Question 32

What tool provides Data Access control, Access Audit, Data Lineage, and Data discovery?

Answer 32

Unity Catalog

Question 33

What is the default trigger interval for structured streaming queries?

Answer 33

half a second

Question 34

How do you establish a trigger to micro-batch data every 5 seconds?

Answer 34

.trigger(processingTime=”5 seconds”)

Question 35

How do you return a GroupedData object?

Answer 35

DataFrame.groupBy()

Question 36

How would you describe a database named db_hr?

Answer 36

DESCRIBE DATABASE db_hr;