Pipeline Design Google Professional Data Engineer GCP

When designing Beam pipeline, consider a few basic questions:

• Where is input data stored? How many sets of input data do you have?
• What does data look like? It might be plaintext, formatted log files, or rows in a database table.
• What do you want to do with data? The core transforms in the Beam SDKs are general purpose.
• What does output data look like, and where should it go?
• Transforms do not consume PCollections