Databricks Certified Data Engineer Associate 시험 - Databricks실제시험문제와 답 - 87문항

Question No : 1

A data engineer needs to create a table in Databricks using data from their organization’s existing SQLite database.
They run the following command:

Which of the following lines of code fills in the above blank to successfully complete the task?

A.org.apache.spark.sql.jdbc
B.autoloader
C.DELTA
D.sqlite
E.org.apache.spark.sql.sqlite

정답:
Explanation:
https://docs.yugabyte.com/preview/integrations/apache-spark/spark-sql/ CREATE TABLE new_employees_table USING JDBC
OPTIONS (
url "<jdbc_url>",
dbtable "<table_name>",
user '<username>',
password '<password>'
) AS
SELECT * FROM employees_table_vw
https://docs.databricks.com/external-data/jdbc.html#language-sql

Question No : 2

A data engineer runs a statement every day to copy the previous day’s sales into the table transactions. Each day’s sales are in their own file in the location "/transactions/raw".
Today, the data engineer runs the following command to complete this task:

After running the command today, the data engineer notices that the number of records in table transactions has not changed.
Which of the following describes why the statement might not have copied any new records into the table?

A.The format of the files to be copied were not included with the FORMAT_OPTIONS keyword.
B.The names of the files to be copied were not included with the FILES keyword.
C.The previous day’s file has already been copied into the table.
D.The PARQUET file format does not support COPY INT
E.The COPY INTO statement requires the table to be refreshed to view the copied rows.

정답:
Explanation:
https://docs.databricks.com/en/ingestion/copy-into/index.html
The COPY INTO SQL command lets you load data from a file location into a Delta table. This is a re-triable and idempotent operation; files in the source location that have already been loaded are skipped. if there are no new records, the only consistent choice is C no new files were loaded because already loaded files were skipped.

Question No : 3

A data analyst has a series of queries in a SQL program. The data analyst wants this program to run every day. They only want the final query in the program to run on Sundays. They ask for help from the data engineering team to complete this task.
Which of the following approaches could be used by the data engineering team to complete this task?

A.They could submit a feature request with Databricks to add this functionality.
B.They could wrap the queries using PySpark and use Python’s control flow system to determine when to run the final query.
C.They could only run the entire program on Sundays.
D.They could automatically restrict access to the source table in the final query so that it is only accessible on Sundays.
E.They could redesign the data model to separate the data used in the final query into a new table.

정답:

Question No : 4

A data engineer needs to apply custom logic to string column city in table stores for a specific use case. In order to apply this custom logic at scale, the data engineer wants to create a SQL user-defined function (UDF).
Which of the following code blocks creates this SQL UDF?
A)

B)

C)

D)

E)

A.Option A
B.Option B
C.Option C
D.Option D
E.Option E

정답:
Explanation:
https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html

Question No : 5

Which of the following commands can be used to write data into a Delta table while avoiding the writing of duplicate records?

A.DROP
B.IGNORE
C.MERGE
D.APPEND
E.INSERT

정답:
Explanation:
To write data into a Delta table while avoiding the writing of duplicate records, you can use the MERGE command. The MERGE command in Delta Lake allows you to combine the ability to insert new records and update existing records in a single atomic operation. The MERGE command compares the data being written with the existing data in the Delta table based on specified matching criteria, typically using a primary key or unique identifier. It then performs conditional actions, such as inserting new records or updating existing records, depending on the comparison results. By using the MERGE command, you can handle the prevention of duplicate records in a more controlled and efficient manner. It allows you to synchronize and reconcile data from different sources while avoiding duplication and ensuring data integrity.

Question No : 6

Which of the following benefits is provided by the array functions from Spark SQL?

A.An ability to work with data in a variety of types at once
B.An ability to work with data within certain partitions and windows
C.An ability to work with time-related data in specified intervals
D.An ability to work with complex, nested data ingested from JSON files
E.An ability to work with an array of tables for procedural automation

정답:
Explanation:
Array functions in Spark SQL are primarily used for working with arrays and complex, nested data structures, such as those often encountered when ingesting JSON files. These functions allow you to manipulate and query nested arrays and structures within your data, making it easier to extract and work with specific elements or values within complex data formats. While some of the other options (such as option A for working with different data types) are features of Spark SQL or SQL in general, array functions specifically excel at handling complex, nested data structures like those found in JSON files.

Question No : 7

A data engineer wants to create a new table containing the names of customers that live in France.
They have written the following command:

A senior data engineer mentions that it is organization policy to include a table property indicating that the new table includes personally identifiable information (PII).
Which of the following lines of code fills in the above blank to successfully complete the task?

A.There is no way to indicate whether a table contains PI
B."COMMENT PII"
C.TBLPROPERTIES PII
D.COMMENT "Contains PII"
E.PII

정답:
Explanation:
Ref: https://www.databricks.com/discover/pages/data-quality-management
CREATE TABLE my_table (id INT COMMENT 'Unique Identification Number', name STRING COMMENT 'PII', age INT COMMENT 'PII') TBLPROPERTIES ('contains_pii'=True) COMMENT 'Contains PII';

Question No : 8

Which of the following commands will return the location of database customer360?

A.DESCRIBE LOCATION customer360;
B.DROP DATABASE customer360;
C.DESCRIBE DATABASE customer360;
D.ALTER DATABASE customer360 SET DBPROPERTIES ('location' = '/user'};
E.USE DATABASE customer360;

정답:
Explanation:
To retrieve the location of a database named "customer360" in a database management system like Hive or Databricks, you can use the DESCRIBE DATABASE command followed by the database name.
This command will provide information about the database, including its location.

Question No : 9

A data analyst has created a Delta table sales that is used by the entire data analysis team. They want help from the data engineering team to implement a series of tests to ensure the data is clean. However, the data engineering team uses Python for its tests rather than SQL.
Which of the following commands could the data engineering team use to access sales in PySpark?

A.SELECT * FROM sales
B.There is no way to share data between PySpark and SQ
C.spark.sql("sales")
D.spark.delta.table("sales")
E.spark.table("sales")

정답:
Explanation:
https://spark.apache.org/docs/3.2.1/api/python/reference/api/pyspark.sql.SparkSession.table.html

Question No : 10

A data engineer has left the organization. The data team needs to transfer ownership of the data engineer’s Delta tables to a new data engineer. The new data engineer is the lead engineer on the data team.
Assuming the original data engineer no longer has access, which of the following individuals must be the one to transfer ownership of the Delta tables in Data Explorer?

A.Databricks account representative
B.This transfer is not possible
C.Workspace administrator
D.New lead data engineer
E.Original data engineer

정답:
Explanation:
https://docs.databricks.com/sql/admin/transfer-ownership.html

Question No : 11

A data engineer needs to determine whether to use the built-in Databricks Notebooks versioning or version their project using Databricks Repos.
Which of the following is an advantage of using Databricks Repos over the Databricks Notebooks versioning?

A.Databricks Repos automatically saves development progress
B.Databricks Repos supports the use of multiple branches
C.Databricks Repos allows users to revert to previous versions of a notebook
D.Databricks Repos provides the ability to comment on specific changes
E.Databricks Repos is wholly housed within the Databricks Lakehouse Platform

정답:
Explanation:
An advantage of using Databricks Repos over the built-in Databricks Notebooks versioning is the ability to work with multiple branches. Branching is a fundamental feature of version control systems like Git, which Databricks Repos is built upon. It allows you to create separate branches for different tasks, features, or experiments within your project. This separation helps in parallel development and experimentation without affecting the main branch or the work of other team members. Branching provides a more organized and collaborative development environment, making it easier to merge changes and manage different development efforts. While Databricks Notebooks versioning also allows you to track versions of notebooks, it may not provide the same level of flexibility and collaboration as branching in Databricks Repos.

Question No : 12

Which of the following data lakehouse features results in improved data quality over a traditional data lake?

A.A data lakehouse provides storage solutions for structured and unstructured data.
B.A data lakehouse supports ACID-compliant transactions.
C.A data lakehouse allows the use of SQL queries to examine data.
D.A data lakehouse stores data in open formats.
E.A data lakehouse enables machine learning and artificial Intelligence workloads.

정답:
Explanation:
One of the key features of a data lakehouse that results in improved data quality over a traditional data lake is its support for ACID (Atomicity, Consistency, Isolation, Durability) transactions. ACID transactions provide data integrity and consistency guarantees, ensuring that operations on the data are reliable and that data is not left in an inconsistent state due to failures or concurrent access. In a traditional data lake, such transactional guarantees are often lacking, making it challenging to maintain data quality, especially in scenarios involving multiple data writes, updates, or complex transformations. A data lakehouse, by offering ACID compliance, helps maintain data quality by providing strong consistency and reliability, which is crucial for data pipelines and analytics.

Question No : 13

Which of the following Git operations must be performed outside of Databricks Repos?

A.Commit
B.Pull
C.Push
D.Clone
E.Merge

정답:
Explanation:
For following tasks, work in your Git provider:
Create a pull request.
Resolve merge conflicts.
Merge or delete branches.
Rebase a branch.
https://docs.databricks.com/repos/index.html

Question No : 14

A data engineer has realized that they made a mistake when making a daily update to a table. They need to use Delta time travel to restore the table to a version that is 3 days old. However, when the data engineer attempts to time travel to the older version, they are unable to restore the data because the data files have been deleted.
Which of the following explains why the data files are no longer present?

A.The VACUUM command was run on the table
B.The TIME TRAVEL command was run on the table
C.The DELETE HISTORY command was run on the table
D.The OPTIMIZE command was nun on the table
E.The HISTORY command was run on the table

정답:
Explanation:
The VACUUM command in Delta Lake is used to clean up and remove unnecessary data files that are no longer needed for time travel or query purposes. When you run VACUUM with certain retention settings, it can delete older data files, which might include versions of data that are older than the specified retention period. If the data engineer is unable to restore the table to a version that is 3 days old because the data files have been deleted, it's likely because the VACUUM command was run on the table, removing the older data files as part of data cleanup.

Question No : 15

Which of the following code blocks will remove the rows where the value in column age is greater than 25 from the existing Delta table my_table and save the updated table?

A.SELECT * FROM my_table WHERE age > 25;
B.UPDATE my_table WHERE age > 25;
C.DELETE FROM my_table WHERE age > 25;
D.UPDATE my_table WHERE age <= 25;
E.DELETE FROM my_table WHERE age <= 25;

정답:

Databricks Databricks Certified Data Engineer Associate 시험