Databricks Pro

Question 1

What does transaction log do?

Answer 1

Records changes to records. Not metadata. Inserts, updates, deletes. Table versioning.

Question 2

What and where is metastore?

Answer 2

high-level metadata about table. Name, schema, location. Info in unity catelog.

Question 3

Where would schema changes go?

Answer 3

Schema changes would be in the metastore and the transaction log.

Question 4

What is Pip and Pypi?

Answer 4

pip is an installation method and pypi is an index that is searched for python packages to install.

Question 5

What is the purpose of a wheel?

Answer 5

It’s a way to package code

Question 6

What are the keywords for a shallow clone?

Answer 6

CREATE TABLE LIKE

Question 7

What are the keywords for a deep clone?

Answer 7

CREATE TABLE AS

Question 8

Which statement regarding static Delta tables in Stream-Static joins is correct?

Answer 8

The latest version of the static Delta table is returned each time it is queried by a microbatch of the stream-static join

Question 9

What statistics are captured in the transaction log?

Answer 9

Total number of records

Minimum value in each column of the first 32 columns of the table

Maximum value in each column of the first 32 columns of the table

Null value counts for in each column of the first 32 columns of the table

Question 10

Databricks secret scope permissions

Answer 10

The secret access permissions are as follows:

MANAGE - Allowed to change ACLs, and read and write to this secret scope.

WRITE - Allowed to read and write to this secret scope.

READ - Allowed to read this secret scope and list what secrets are available.

Question 11

do

What does sys.path do?

Answer 11

The sys.path variable contains a list of directories where the Python interpreter searches for modules.

Question 12

What is DBFS

Answer 12

Abstraction on top of scalable object storage like ADLS.

Question 13

Can attach permission

Answer 13

Attach to cluster, view UI, and view metrics

Question 14

Attach to cluster, view UI, and view metrics, terminate as well as start and restart compute

Answer 14

can restart permission

Question 15

cluster permission that can do everything. For example, editing.

Answer 15

can manage

Question 16

MLFlow model

Answer 16

Mlflow.pyfunc.spark_udf function allows to register a MLFlow model as a Apache Spark UDF.

The registration allows you to use the trained model within Spark, enabling you to apply the model to new data in a distributed manner.

Question 17

OPTIMIZE file size

Answer 17

1 GB

Question 18

Auto Optimize file size

Answer 18

128 MB

Question 19

Does auto-compact support z-ordering?

Answer 19

No

Question 20

What’s the minimum size for a partition?

Answer 20

1 GB

Question 21

Spark UI

Answer 21

So a compute in databricks is reading and writing data. I think Input Size means reading data. So does Min: 78 KiB / 16 records mean that the minimum amount of data read was 16 records, which amounted to 78 KiB in space?