Google Data Engineer Sample Questions:

1. You have a data pipeline with a Cloud Dataflow job that aggregates and writes time series metrics to Cloud Bigtable. This data feeds a dashboard used by thousands of users across the organization. You need to support additional concurrent users and reduce the amount of time required to write the dat a. Which two actions should you take? (Choose two.)

A) Increase the number of nodes in the Cloud Bigtable cluster
B) Modify your Cloud Dataflow pipeline to use the Flatten transform before writing to Cloud Bigtable
C) Configure your Cloud Dataflow pipeline to use local execution
D) Modify your Cloud Dataflow pipeline to use the CoGroupByKey transform before writing to Cloud Bigtable
E) Increase the maximum number of Cloud Dataflow workers by setting maxNumWorkers in PipelineOptions

2. You are implementing several batch jobs that must be executed on a schedule. These jobs have many interdependent steps that must be executed in a specific order. Portions of the jobs involve executing shell scripts, running Hadoop jobs, and running queries in BigQuery. The jobs are expected to run for many minutes up to several hours. If the steps fail, they must be retried a fixed number of times. Which service should you use to manage the execution of these jobs?

A) Cloud Dataflow
B) Cloud Scheduler
C) Cloud Composer
D) Cloud Functions

3. Flowlogistic's management has determined that the current Apache Kafka servers cannot handle the data volume for their real-time inventory tracking system. You need to build a new system on Google Cloud Platform (GCP) that will feed the proprietary tracking software. The system must be able to ingest data from a variety of global sources, process and query in real-time, and store the data reliably. Which combination of GCP products should you choose?

A) Cloud Pub/Sub, Cloud Dataflow, and Local SSD
B) Cloud Pub/Sub, Cloud SQL, and Cloud Storage
C) Cloud Pub/Sub, Cloud Dataflow, and Cloud Storage
D) Cloud Load Balancing, Cloud Dataflow, and Cloud Storage

4. You want to automate execution of a multi-step data pipeline running on Google Cloud. The pipeline includes Cloud Dataproc and Cloud Dataflow jobs that have multiple dependencies on each other. You want to use managed services where possible, and the pipeline will run every day. Which tool should you use?

A) Workflow Templates on Cloud Dataproc
B) Cloud Scheduler
C) Cloud Composer
D) cron

5. Which Cloud Dataflow / Beam feature should you use to aggregate data in an unbounded data source every hour based on the time when the data entered the pipeline?

A) A processing time trigger
B) The with Allowed Lateness method
C) An hourly watermark
D) An event time trigger

Solutions: