Download Databricks - Data Engineering Associate Certification Study Guide -exam elaborations with and more Exams Engineering in PDF only on Docsity!
Databricks - Data Engineering
Associate Certification Study Guide
-exam elaborations with 100% verified
answer/solutions- Excel & Succeed - academic
year (2026-2027)
45 Q&A
Which of the following describes a benefit of a data lakehouse that is unavailable in a traditional data warehouse? A. A data lakehouse provides a relational system of data management. B. A data lakehouse captures snapshots of data for version control purposes. C. A data lakehouse couples storage and compute for complete control. D. A data lakehouse utilizes proprietary storage formats for data. E. A data lakehouse enables both batch and streaming analytics. E. A data lakehouse enables both batch and streaming analytics Which of the following locations hosts the driver and worker nodes of a Databricks-managed cluster? A. Data plane B. Control plane C. Databricks Filesystem
D. JDBC data source E. Databricks web application A. Data Plane A data architect is designing a data model that works for both video- based machine learning workloads and highly audited batch ETL/ELT workloads. Which of the following describes how using a data lakehouse can help the data architect meet the needs of both workloads? A. A data lakehouse requires very little data modeling. B. A data lakehouse combines compute and storage for simple governance. C. A data lakehouse provides autoscaling for compute clusters. D. A data lakehouse stores unstructured data and is ACID-compliant. E. A data lakehouse fully exists in the cloud D. A data lakehouse stores unstructured data and is ACID-compliant. Which of the following describes a scenario in which a data engineer will want to use a Job cluster instead of an all-purpose cluster? A. An ad-hoc analytics report needs to be developed while minimizing compute costs. B. A data team needs to collaborate on the development of a machine learning model. C. An automated workflow needs to be run every 30 minutes. D. A Databricks SQL query needs to be scheduled for upward reporting.
comments D. Databricks Notebooks support the use of multiple languages in the same notebook E. Databricks Notebooks support the creation of interactive data visualizations B. Databricks Notebooks support real-time coauthoring on a single notebook Which of the following describes how Databricks Repos can help facilitate CI/CD workflows on the Databricks Lakehouse Platform? A. Databricks Repos can facilitate the pull request, review, and approval process before merging branches B. Databricks Repos can merge changes from a secondary Git branch into a main Git branch C. Databricks Repos can be used to design, develop, and trigger Git automation pipelines D. Databricks Repos can store the single-source-of-truth Git repository E. Databricks Repos can commit or push code changes to trigger a CI/CD process E. Databricks Repos can commit or push code changes to trigger a CI/CD process Which of the following statements describes Delta Lake? A. Delta Lake is an open source analytics engine used for big data workloads. B. Delta Lake is an open format storage layer that delivers reliability, security, and performance. C. Delta Lake is an open source platform to help manage the complete machine learning lifecycle.
D. Delta Lake is an open source data storage format for distributed data. E. Delta Lake is an open format storage layer that processes data. B. Delta Lake is an open format storage layer that delivers reliability, security, and performance. A data architect has determined that a table of the following format is necessary: id birthDate avgRating a1 1990-01-06 5. a2 1974-11-21 7. ... ... ... Which of the following code blocks uses SQL DDL commands to create an empty Delta table in the above format regardless of whether a table already exists with this name? A. CREATE OR REPLACE TABLE table_name AS SELECT id STRING, birthDate DATE, avgRating FLOAT USING DELTA B. CREATE OR REPLACE TABLE table_name ( id STRING, birthDate DATE, avgRating FLOAT ) C. CREATE TABLE IF NOT EXISTS table_name ( id STRING, birthDate DATE, avgRating FLOAT ) D. CREATE TABLE table_name AS
C. Bin-packing D. Write as a Parquet file E. Tuning the file size B. Z-Ordering A data engineer needs to create a database called customer360 at the location /customer/customer360. The data engineer is unsure if one of their colleagues has already created the database. Which of the following commands should the data engineer run to complete this task? A. CREATE DATABASE customer360 LOCATION '/customer/customer360'; B. CREATE DATABASE IF NOT EXISTS customer360; C. CREATE DATABASE IF NOT EXISTS customer360 LOCATION '/customer/customer360'; D. CREATE DATABASE IF NOT EXISTS customer360 DELTA LOCATION '/customer/customer360'; E. CREATE DATABASE customer360 DELTA LOCATION '/customer/customer360'; C. CREATE DATABASE IF NOT EXISTS customer360 LOCATION '/customer/customer360'; A junior data engineer needs to create a Spark SQL table my_table for which Spark manages both the data and the metadata. The metadata and data should also be stored in the Databricks Filesystem (DBFS). Which of the following commands should a senior data engineer share with the junior data engineer to complete this task? A. CREATE TABLE my_table (id STRING, value STRING) USING
org.apache.spark.sql.parquet OPTIONS (PATH "storage-path"); B. CREATE MANAGED TABLE my_table (id STRING, value STRING) USING org.apache.spark.sql.parquet OPTIONS (PATH "storage-path"); C. CREATE MANAGED TABLE my_table (id STRING, value STRING); D. CREATE TABLE my_table (id STRING, value STRING) USING DBFS; E. CREATE TABLE my_table (id STRING, value STRING); E. CREATE TABLE my_table (id STRING, value STRING); A data engineer wants to create a relational object by pulling data from two tables. The relational object must be used by other data engineers in other sessions. In order to save on storage costs, the data engineer wants to avoid copying and storing physical data. Which of the following relational objects should the data engineer create? A. View B. Temporary view C. Delta Table D. Database E. Spark SQL Table A. View A data engineering team has created a series of tables using Parquet data stored in an external system. The team is noticing that after appending new rows to the data in the external system, their queries within Databricks are not returning the new rows. They identify the caching of the previous data as the cause of this issue. Which of the following approaches will ensure that the data returned by
B. CREATE TABLE AS SELECT statements infer the schema by scanning the data. C. CREATE TABLE AS SELECT statements result in tables where schemas are optional. D. CREATE TABLE AS SELECT statements assign all columns the type STRING. E. CREATE TABLE AS SELECT statements result in tables that do not support schemas. A. CREATE TABLE AS SELECT statements adopt schema details from the source table and query. A data engineer is overwriting data in a table by deleting the table and recreating the table. Another data engineer suggests that this is inefficient and the table should simply be overwritten instead. Which of the following reasons to overwrite the table instead of deleting and recreating the table is incorrect? A. Overwriting a table is efficient because no files need to be deleted. B. Overwriting a table results in a clean table history for logging and audit purposes. C. Overwriting a table maintains the old version of the table for Time Travel. D. Overwriting a table is an atomic operation and will not leave the table in an unfinished state. E. Overwriting a table allows for concurrent queries to be completed while in progress. B. Overwriting a table results in a clean table history for logging and audit purposes.
Which of the following commands will return records from an existing Delta table my_table where duplicates have been removed? A. DROP DUPLICATES FROM my_table; B. SELECT * FROM my_table WHERE duplicate = False; C. SELECT DISTINCT * FROM my_table; D. MERGE INTO my_table a USING new_records b ON a.id = b.id WHEN NOT MATCHED THEN INSERT *; E. MERGE INTO my_table a USING new_records b; C. SELECT DISTINCT * FROM my_table; A data engineer wants to horizontally combine two tables as a part of a query. They want to use a shared column as a key column, and they only want the query result to contain rows whose value in the key column is present in both tables. Which of the following SQL commands can they use to accomplish this task? A. INNER JOIN B. OUTER JOIN C. LEFT JOIN D. MERGE E. UNION A. INNER JOIN A junior data engineer has ingested a JSON file into a table raw_table with the following schema: cart_id STRING,
A. SELECT transaction_id, explode(payload) FROM raw_table; B. SELECT transaction_id, payload.date FROM raw_table; C. SELECT transaction_id, date FROM raw_table; D. SELECT transaction_id, payload[date] FROM raw_table; E. SELECT transaction_id, date from payload FROM raw_table; B. SELECT transaction_id, payload.date FROM raw_table; Question 22 goes past the number of allowed characters!!! Look at pdf!! A data analyst has provided a data engineering team with the following Spark SQL query: SELECT district, avg(sales) FROM store_sales_ GROUP BY district; The data analyst would like the data engineering team to run this query every day. The date at the end of the table name (20220101) should automatically be replaced with the current date each time the query is run. Which of the following approaches could be used by the data engineering team to efficiently automate this process? A. They could wrap the query using PySpark and use Python's string variable system to automatically update the table name. B. They could manually replace the date within the table name with the current day's date. C. They could request that the data analyst rewrites the query to be run less frequently. D. They could replace the string-formatted date in A. They could wrap the query using PySpark and use Python's string variable system to automatically update the table name. A data engineer has ingested data from an external source into a PySpark DataFrame raw_df. They need to briefly make this data available in SQL for a data analyst to perform a quality assurance check on the
data. Which of the following commands should the data engineer run to make this data available in SQL for only the remainder of the Spark session? A. raw_df.createOrReplaceTempView("raw_df") B. raw_df.createTable("raw_df") C. raw_df.write.save("raw_df") D. raw_df.saveAsTable("raw_df") E. There is no way to share data between PySpark and SQL. A. raw_df.createOrReplaceTempView("raw_df") A data engineer needs to dynamically create a table name string using three Python variables: region, store, and year. An example of a table name is below when region = "nyc", store = "100", and year = "2021": nyc100_sales_ Which of the following commands should the data engineer use to construct the table name in Python? A. "{region}+{store}+sales+{year}" B. f"{region}+{store}+sales+{year}" C. "{region}{store}sales{year}" D. f"{region}{store}sales{year}" E. {region}+{store}+"sales"+{year} D. f"{region}{store}sales{year}" A data engineer has developed a code block to perform a streaming read on a data source. The code block is below:
If the data engineer only wants the query to execute a single micro-batch to process all of the available data, which of the following lines of code should the data engineer use to fill in the blank? A. trigger(once=True) B. trigger(continuous="once") C. processingTime("once") D. trigger(processingTime="once") E. processingTime(1) A. trigger(once=True) A data engineer is designing a data pipeline. The source system generates files in a shared directory that is also used by other processes. As a result, the files should be kept as is and will accumulate in the directory. The data engineer needs to identify which files are new since the previous run in the pipeline, and set up the pipeline to only ingest those new files with each run. Which of the following tools can the data engineer use to solve this problem? A. Databricks SQL B. Delta Lake C. Unity Catalog D. Data Explorer E. Auto Loader E. Auto Loader
A data engineering team is in the process of converting their existing data pipeline to utilize Auto Loader for incremental processing in the ingestion of JSON files. One data engineer comes across the following code block in the Auto Loader documentation: (streaming_df = spark.readStream.format("cloudFiles") .option("cloudFiles.format", "json") .option("cloudFiles.schemaLocation", schemaLocation) .load(sourcePath)) Assuming that schemaLocation and sourcePath have been set correctly, which of the following changes does the data engineer need to make to convert this code block to use Auto Loader to ingest the data? A. The data engineer needs to change the format("cloudFiles") line to format("autoLoader"). B. There is no change required. Databricks automatically uses Auto Loader for streaming reads. C. There is no change required. The inclusion of format("cloudFiles") enables the use of Auto Loade C. There is no change required. The inclusion of format("cloudFiles") enables the use of Auto Loader. Which of the following data workloads will utilize a Bronze table as its source? A. A job that aggregates cleaned data to create standard summary statistics B. A job that queries aggregated data to publish key insights into a dashboard C. A job that ingests raw data from a streaming source into the Lakehouse D. A job that develops a feature set for a machine learning application
col("units")) .writeStream .option("checkpointLocation", checkpointPath) .outputMode("append") .table("cleanedSales") ) D. (spark.readStream.load(rawSalesLocation) .writeStream .option("checkp ointLocation", checkpointPath) .outputMode("append") .table("uncleanedSales") ) E. (spark.read.load(rawSalesLocation) .writeStream .option("checkpointLoc ation", checkpoin C. (spark.table("sales") .withColumn("avgPrice", col("sales") / col("units")) .writeStream .option("checkpointLocation",checkpointPath) .outputMode("append") .table("cleanedSales") ) Which of the following benefits does Delta Live Tables provide for ELT pipelines over standard data pipelines that utilize Spark and Delta Lake on Databricks? A. The ability to declare and maintain data table dependencies B. The ability to write pipelines in Python and/or SQL C. The ability to access previous versions of data tables D. The ability to automatically scale compute resources E. The ability to perform batch and streaming queries A. The ability to declare and maintain data table dependencies A data engineer has three notebooks in an ELT pipeline. The notebooks need to be executed in a specific order for the pipeline to complete successfully. The data engineer would like to use Delta Live Tables to manage this process. Which of the following steps must the data engineer take as part of
implementing this pipeline using Delta Live Tables? A. They need to create a Delta Live Tables pipeline from the Data page. B. They need to create a Delta Live Tables pipeline from the Jobs page. C. They need to create a Delta Live tables pipeline from the Compute page. D. They need to refactor their notebook to use Python and the dlt library. E. They need to refactor their notebook to use SQL and CREATE LIVE TABLE keyword. B. They need to create a Delta Live Tables pipeline from the Jobs page. A data engineer has written the following query: SELECT * FROM json./path/to/json/file.json; The data engineer asks a colleague for help to convert this query for use in a Delta Live Tables (DLT) pipeline. The query should create the first table in the DLT pipeline. Which of the following describes the change the colleague needs to make to the query? A. They need to add a COMMENT line at the beginning of the query. B. They need to add a CREATE LIVE TABLE table_name AS line at the beginning of the query. C. They need to add a live. prefix prior to json. in the FROM line. D. They need to add a CREATE DELTA LIVE TABLE table_name AS line at the beginning of the query. E. They need to add the cloud_files(...) wrapper to the JSON file path.
