Dataflow templates allow you to package a Dataflow pipeline for deployment. Anyone with the correct permissions can then use the template to deploy the packaged pipeline. You can create your own custom Dataflow templates, and Google provides pre-built templates for common scenarios.

Worflow

Using Dataflow templates involves the following high-level steps:

1. Developers set up a development environment and develop their pipeline. The environment includes the Apache Beam SDK and other dependencies.
2. Depending on the template type (Flex or classic):
   • For Flex templates, the developers package the pipeline into a Docker image, push the image to Artifact Registry, and upload a template specification file to Cloud Storage.
   • For classic templates, developers run the pipeline, create a template file, and stage the template to Cloud Storage.
3. Other users submit a request to the Dataflow service to run the template.
4. Dataflow creates a pipeline from the template. The pipeline can take as much as five to seven minutes to start running.