The following is guidance on how to practically make changes that fix workflows.
-
Create a branch in https://github.com/apache/beam not your fork.
The reason to perform changes to a branch of the main repo instead of your fork is due to the challenge in replicating the environment within which Beam GitHub workflows execute. GitHub workflows allow you to execute against a branch of a repo.
-
Make changes in this branch you anticipate will fix the failing workflow.
-
Run the workflow designating your branch.
In the GitHub workflow interface, you can designate any branch of the repository to run the workflow against. Selecting your branch allows you to test the changes you made. The following screenshot shows an example of this feature.
-
Create a PR, pasting the link to your successful workflow run in the branch
When doing a PR, the checks will not run against your branch. Your reviewer may not know this so you'll want to mention this in your PR description, pasting the link to your successful run.
-
After PR merges, execute the workflow manually to validate your merged changes.
Most workflows will get kicked off automatically when you open a PR, push code, or on a schedule.
If you would like to manually trigger a job, you have 2 options:
-
Trigger Phrases: Some jobs have trigger phrases associated with them (e.g.
Run XYZ PreCommit). These will appear in statuses of previous PR runs of that check. You can trigger the job on any PR by commenting that trigger phrase in the PR.Note: this approach is found not scalable (#28909) and currently only enabled for PreCommit workflows. For PostCommit jobs, it is currently replaced by a temporary approach: test suites are configured to trigger whenever a particular trigger file is modified. Test workflows have pull_request_target paths, which include a trigger file. Whenever a trigger file is modified, the test suite will trigger on the pull request. Make any change to this file to trigger the job. The trigger file looks like the following:
.github/trigger_files/<workflow_file_name.json>. -
Committers only - Manual triggering: Any committer can start any job with a workflow_dispatch trigger defined (all jobs should have these). To do so, navigate to the Actions tab, click on your desired workflow in the left navigation bar, and then click
Run Workflow.
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
- Name and phrase set via matrix for re-run to work (See below)
- A set of specific triggers
- An explicit checkout step
- A set of GitHub token permissions
- Concurrency Groups
- Comment Triggering Support
Each of these is described in more detail below.
Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
jobs:
beam_job_name:
name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
strategy:
matrix:
job_name: [beam_job_name]
job_phrase: [Run Job Phrase]
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
github.event_name == 'workflow_dispatch' ||
github.event.comment.body == 'Run Job Phrase'
steps:
- uses: actions/checkout@v3
- name: Setup repository
uses: ./.github/actions/setup-action
with:
comment_phrase: ${{ matrix.job_phrase }}
github_token: ${{ secrets.GITHUB_TOKEN }}
github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
And in case when the workflow already utilizes matrix do the following:
jobs:
beam_job_with_matrix:
name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
runs-on: [self-hosted, ubuntu-20.04, main]
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
job_name: ["beam_job_with_matrix"]
job_phrase: ["Run Job With Matrix"]
python_version: ['3.8','3.9','3.10','3.11']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
github.event_name == 'workflow_dispatch' ||
startsWith(github.event.comment.body, 'Run Job With Matrix')
steps:
- uses: actions/checkout@v3
- name: Setup repository
uses: ./.github/actions/setup-action
with:
comment_phrase: ${{ matrix.job_phrase }}
github_token: ${{ secrets.GITHUB_TOKEN }}
github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
Notice how the matrix element of python_version is appended to the name.
GitHub allows workflows to define a set of triggers that dictate when a job should be run. For more info, see documentation here.
For the purposes of Beam, each CI workflow should define the following triggers:
- A
pushtrigger - A
pull_request_targettrigger - An issue_comment trigger (for issue created). This is needed for comment triggering support (see section below).
- A scheduled trigger
- A workflow_dispatch trigger
The push/pull_request_target triggers should only run when appropriate paths are modified. See https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Go.yml#L4 for an example (you can copy and paste this into your workflow, you just need to change the paths).
Because we use the pull_request_target trigger instead of pull_request, we need an explicit checkout of the correct commit. This can be done as a step that uses the setup-action action in your workflow. See
beam/.github/workflows/beam_PreCommit_Go.yml
Lines 65 to 70 in 0ee2dc7
You should explicitly define the GitHub Actions token scopes needed by your job. For most jobs, this will be actions: write (needed for comment triggering support) and read permissions for all other options. See
Concurrency groups are a way of making sure that no more than one Actions run is running per job/group at any given time (previous ones can be cancelled). To reduce resource usage, you should define the following concurrency group:
concurrency:
group: '${{ github.workflow }} @ ${{ github.event.issue.number || github.sha || github.head_ref || github.ref }}-${{ github.event.schedule || github.event.comment.id || github.event.sender.login }}'
cancel-in-progress: true
this defines the following groups:
- Each workflow will represent a different set of groups
- Within each workflow, each PR will represent a single group
- Within each non-PR run for a workflow, each commit will represent a set of groups
- Within each commit, each push event, schedule event, and manual run event will represent a set of groups
In order to make it easier for non-committers to interact with workflows, workflows should implement comment triggering support. This requires 3 additional components beyond the ones mentioned above:
- Each job should be gated on an if condition that maps to that workflow's comment trigger (example: )
- Each job should have the rerun action immediately after its checkout step. You can add a step that uses the
setup-actionaction in your workflow, which encapsulates the checkout and rerun logic in one place. This should be gated on the comment trigger (example:)beam/.github/workflows/beam_PreCommit_Go.yml
Lines 65 to 70 in 0ee2dc7
- Each job should have a descriptive name that includes the comment trigger (example: )
Note: Comment triggering is found not scalable (#28909) and is currently limited to a subset of suites. For more information see the Running Workflows Manually section.
New workflows are not added to the UI on the Actions tab until they are merged into the repo's main branch (master for Beam). To test new workflows, we recommend the following pattern:
- Fork the Beam repo
- Add your proposed workflow to the main branch of your fork.
- Run the workflow in the Actions tab of your fork using the
Run workflowbutton.
Note: most workflows use self-hosted runners
with the main and ubuntu labels to execute (example).
If you are testing on a fork, you likely will not have self-hosted runners set up.
To work around this, you can start using hosted runners and then switch over when you're ready to create a PR.
You can do this by changing runs-on: [self-hosted, ubuntu-20.04, main] (self-hosted, use in your PR) to runs-on: ubuntu-20.04 (GitHub hosted, use for local testing).
Note when using ubuntu-20.04 as the host, you might need to choose the Java version since some gradle tasks only work with a certain Java version.
One example is below to use Java 11 when testing your workflow:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '11'
If you need to make more changes to the workflow yaml file after it has been added to the repo, you can develop normally on a branch (using your fork or the main Beam repo if you are a committer). If you are a non-committer, this flow has several caveats mentioned below. To do this:
- Make your change on a development branch.
- Navigate to your workflow in the Actions tab. If your changes are on a fork, navigate to the fork's Actions tab instead. If you don't see the correct action, make sure that your fork's main branch is up to date with Beam's master branch.
- Click run workflow. Before clicking submit, update to run on your branch.
Note: If you run a workflow from your fork of Beam, it will not have access to secrets stored in the Beam repository. This will cause some things like authenticating to external services to fail, which may cause your workflow to fail. If you run into this issue, you can either:
- Ask for a committers help to add the workflow to a branch on the main apache/beam repo.
- Upload secrets to your repo mirroring the secrets used in the main Beam repo.
- Wait until the changes are merged into Beam to test (this should only be done rarely).
Additionally, as mentioned above your fork likely will not have self-hosted runners set up. To work around this, you can start using hosted runners and then switch over when you're ready to create a PR. You can do this by changing runs-on: [self-hosted, ubuntu-20.04, main] (self-hosted, use in your PR) to runs-on: ubuntu-20.04 (GitHub hosted, use for local testing).
Please note that jobs with matrix need to have matrix element in the comment. Example:
Run Python PreCommit (3.8)
PreCommit Jobs run in a schedule and also get triggered in a PR if relevant sources has changed. To manually trigger certain PreCommit job, comment with the Trigger Phrase (listed below) in the PR.
| Workflow name | Matrix | Trigger Phrase | Cron Status |
|---|---|---|---|
| PreCommit Community Metrics | N/A | Run CommunityMetrics PreCommit |
 |
| PreCommit GHA | N/A | Run GHA PreCommit |
 |
| PreCommit Go | N/A | Run Go PreCommit |
 |
| PreCommit GoPortable | N/A | Run GoPortable PreCommit |
 |
| PreCommit Java | N/A | Run Java PreCommit |
 |
| PreCommit Java Amazon Web Services2 IO Direct | N/A | Run Java_Amazon-Web-Services2_IO_Direct PreCommit |
 |
| PreCommit Java Amqp IO Direct | N/A | Run Java_Amqp_IO_Direct PreCommit |
 |
| PreCommit Java Azure IO Direct | N/A | Run Java_Azure_IO_Direct PreCommit |
 |
| PreCommit Java Cassandra IO Direct | N/A | Run Java_Cassandra_IO_Direct PreCommit |
 |
| PreCommit Java Cdap IO Direct | N/A | Run Java_Cdap_IO_Direct PreCommit |
 |
| PreCommit Java Clickhouse IO Direct | N/A | Run Java_Clickhouse_IO_Direct PreCommit |
 |
| PreCommit Java Csv IO Direct | N/A | Run Java_Csv_IO_Direct PreCommit |
 |
| PreCommit Java Debezium IO Direct | N/A | Run Java_Debezium_IO_Direct PreCommit |
 |
| PreCommit Java ElasticSearch IO Direct | N/A | Run Java_ElasticSearch_IO_Direct PreCommit |
 |
| PreCommit Java Examples Dataflow | N/A | Run Java_Examples_Dataflow PreCommit |
 |
| PreCommit Java Examples Dataflow Java21 | N/A | Run Java_Examples_Dataflow_Java21 PreCommit |
 |
| PreCommit Java File-schema-transform IO Direct | N/A | Run Java_File-schema-transform_IO_Direct PreCommit |
 |
| PreCommit Java Flink Versions | N/A | Run Java_Flink_Versions PreCommit |
 |
| PreCommit Java GCP IO Direct | N/A | Run Java_GCP_IO_Direct PreCommit |
 |
| PreCommit Java Google-ads IO Direct | N/A | Run Java_Google-ads_IO_Direct PreCommit |
 |
| PreCommit Java Hadoop IO Direct | N/A | Run Java_Hadoop_IO_Direct PreCommit |
 |
| PreCommit Java HBase IO Direct | N/A | Run Java_HBase_IO_Direct PreCommit |
 |
| PreCommit Java HCatalog IO Direct | N/A | Run Java_HCatalog_IO_Direct PreCommit |
 |
| PreCommit Java Kafka IO Direct | N/A | Run Java_Kafka_IO_Direct PreCommit |
 |
| PreCommit Java InfluxDb IO Direct | N/A | Run Java_InfluxDb_IO_Direct PreCommit |
 |
| PreCommit Java IOs Direct | N/A | Run Java_IOs_Direct PreCommit |
N/A |
| PreCommit Java JDBC IO Direct | N/A | Run Java_JDBC_IO_Direct PreCommit |
 |
| PreCommit Java Jms IO Direct | N/A | Run Java_Jms_IO_Direct PreCommit |
 |
| PreCommit Java Kudu IO Direct | N/A | Run Java_Kudu_IO_Direct PreCommit |
 |
| PreCommit Java MongoDb IO Direct | N/A | Run Java_MongoDb_IO_Direct PreCommit |
 |
| PreCommit Java Mqtt IO Direct | N/A | Run Java_Mqtt_IO_Direct PreCommit |
 |
| PreCommit Java Neo4j IO Direct | N/A | Run Java_Neo4j_IO_Direct PreCommit |
 |
| PreCommit Java Parquet IO Direct | N/A | Run Java_Parquet_IO_Direct PreCommit |
 |
| PreCommit Java Pulsar IO Direct | N/A | Run Java_Pulsar_IO_Direct PreCommit |
 |
| PreCommit Java PVR Flink Batch | N/A | Run Java_PVR_Flink_Batch PreCommit |
 |
| PreCommit Java PVR Flink Docker | N/A | Run Java_PVR_Flink_Docker PreCommit |
 |
| PreCommit Java RabbitMq IO Direct | N/A | Run Java_RabbitMq_IO_Direct PreCommit |
 |
| PreCommit Java Redis IO Direct | N/A | Run Java_Redis_IO_Direct PreCommit |
 |
| PreCommit Java RequestResponse IO Direct | N/A | Run Java_RequestResponse_IO_Direct PreCommit |
 |
| PreCommit Java SingleStore IO Direct | N/A | Run Java_SingleStore_IO_Direct PreCommit |
 |
| PreCommit Java Snowflake IO Direct | N/A | Run Java_Snowflake_IO_Direct PreCommit |
 |
| PreCommit Java Solr IO Direct | N/A | Run Java_Solr_IO_Direct PreCommit |
 |
| PreCommit Java Spark3 Versions | N/A | Run Java_Spark3_Versions PreCommit |
 |
| PreCommit Java Splunk IO Direct | N/A | Run Java_Splunk_IO_Direct PreCommit |
 |
| PreCommit Java Thrift IO Direct | N/A | Run Java_Thrift_IO_Direct PreCommit |
 |
| PreCommit Java Tika IO Direct | N/A | Run Java_Tika_IO_Direct PreCommit |
 |
| PreCommit Kotlin Examples | N/A | Run Kotlin_Examples PreCommit |
 |
| PreCommit Portable Python | ['3.8','3.11'] | Run Portable_Python PreCommit |
 |
| PreCommit Python | ['3.8','3.9','3.10','3.11'] | Run Python PreCommit (matrix_element) |
 |
| PreCommit Python Coverage | N/A | Run Python_Coverage PreCommit |
 |
| PreCommit Python Dataframes | ['3.8','3.9','3.10','3.11'] | Run Python_Dataframes PreCommit (matrix_element) |
 |
| PreCommit Python Docker | ['3.8','3.9','3.10','3.11'] | Run PythonDocker PreCommit (matrix_element) |
 |
| PreCommit Python Docs | N/A | Run PythonDocs PreCommit |
 |
| PreCommit Python Examples | ['3.8','3.9','3.10','3.11'] | Run Python_Examples PreCommit (matrix_element) |
 |
| PreCommit Python Formatter | N/A | Run PythonFormatter PreCommit |
 |
| PreCommit Python Integration | ['3.8','3.11'] | Run Python_Integration PreCommit (matrix_element) |
 |
| PreCommit Python Lint | N/A | Run PythonLint PreCommit |
 |
| PreCommit Python ML | ['3.8','3.9','3.10','3.11'] | Run Python_ML PreCommit (matrix_element) |
 |
| PreCommit Python PVR Flink | N/A | Run Python_PVR_Flink PreCommit |
 |
| PreCommit Python Runners | ['3.8','3.9','3.10','3.11'] | Run Python_Runners PreCommit (matrix_element) |
 |
| PreCommit Python Transforms | ['3.8','3.9','3.10','3.11'] | Run Python_Transforms PreCommit (matrix_element) |
 |
| PreCommit RAT | N/A | Run RAT PreCommit |
 |
| PreCommit Spotless | N/A | Run Spotless PreCommit |
 |
| PreCommit SQL | N/A | Run SQL PreCommit |
 |
| PreCommit SQL Java8 | N/A | Run SQL_Java8 PreCommit |
 |
| PreCommit SQL Java17 | N/A | Run SQL_Java17 PreCommit |
 |
| PreCommit Typescript | N/A | Run Typescript PreCommit |
 |
| PreCommit Website | N/A | Run Website PreCommit |
 |
| PreCommit Website Stage GCS | N/A | Run Website_Stage_GCS PreCommit |
 |
| PreCommit Whitespace | N/A | Run Whitespace PreCommit |
 |
| PreCommit Xlang Generated Transforms | N/A | Run Xlang_Generated_Transforms PreCommit |
 |
| PreCommit YAML Xlang Direct | N/A | Run Yaml_Xlang_Direct PreCommit |
 |
Additional PreCommit jobs running basic SDK unit test on a matrices of operating systems. These workflows were setup differently and currently do not support trigger phrases
| Workflow name | Cron Status |
|---|---|
| Go Tests |  |
| Java Tests |  |
| Python Tests |  |
| TypeScript Tests |  |
| Build Wheels |  |
PostCommit Jobs run in a schedule against master branch and generally do not get triggered in a PR. To manually trigger certain PostCommit job, attach a file named "Trigger file" (listed below) under .github/trigger_files/ folder.
| Workflow name | Matrix | Trigger file | Cron Status |
|---|---|---|---|
| PostCommit Go | N/A | beam_PostCommit_Go.json |
 |
| PostCommit Go Dataflow ARM | N/A | beam_PostCommit_Go_Dataflow_ARM.json |
 |
| PostCommit Go VR Flink | N/A | beam_PostCommit_Go_VR_Flink.json |
 |
| PostCommit Go VR Samza | N/A | beam_PostCommit_Go_VR_Samza.json |
 |
| PostCommit Go VR Spark | N/A | beam_PostCommit_Go_VR_Spark.json |
 |
| PostCommit Java Avro Versions | N/A | beam_PostCommit_Java_Avro_Versions.json |
 |
| PostCommit Java BigQueryEarlyRollout | N/A | beam_PostCommit_Java_BigQueryEarlyRollout.json |
 |
| PostCommit Java Dataflow V1 | N/A | beam_PostCommit_Java_DataflowV1.json |
 |
| PostCommit Java Dataflow V2 | N/A | beam_PostCommit_Java_DataflowV2.json |
 |
| PostCommit Java Examples Dataflow ARM | ['8','11','17','21'] | beam_PostCommit_Java_Examples_Dataflow_ARM.json |
 |
| PostCommit Java Examples Dataflow | N/A | beam_PostCommit_Java_Examples_Dataflow.json |
 |
| PostCommit Java Examples Dataflow Java | ['8','17','21'] | beam_PostCommit_Java_Examples_Dataflow_Java.json |
 |
| PostCommit Java Examples Dataflow V2 | N/A | beam_PostCommit_Java_Examples_Dataflow_V2.json |
 |
| PostCommit Java Examples Dataflow V2 Java | ['8','17','21'] | beam_PostCommit_Java_Examples_Dataflow_V2_Java.json |
 |
| PostCommit Java Examples Direct | N/A | beam_PostCommit_Java_Examples_Direct.json |
 |
| PostCommit Java Examples Flink | N/A | beam_PostCommit_Java_Examples_Flink.json |
 |
| PostCommit Java Examples Spark | N/A | beam_PostCommit_Java_Examples_Spark.json |
 |
| PostCommit Java Hadoop Versions | N/A | beam_PostCommit_Java_Hadoop_Versions.json |
 |
| PostCommit Java IO Performance Tests | N/A | beam_PostCommit_Java_IO_Performance_Tests.json |
 |
| PostCommit Java InfluxDbIO Integration Test | N/A | beam_PostCommit_Java_InfluxDbIO_IT.json |
 |
| PostCommit Java Jpms Dataflow Java11 | N/A | beam_PostCommit_Java_Jpms_Dataflow_Java11.json |
 |
| PostCommit Java Jpms Dataflow Java17 | N/A | beam_PostCommit_Java_Jpms_Dataflow_Java17.json |
 |
| PostCommit Java Jpms Direct Java11 | N/A | beam_PostCommit_Java_Jpms_Direct_Java11.json |
 |
| PostCommit Java Jpms Direct Java17 | N/A | beam_PostCommit_Java_Jpms_Direct_Java17.json |
 |
| PostCommit Java Jpms Direct Java21 | N/A | beam_PostCommit_Java_Jpms_Direct_Java21.json |
 |
| PostCommit Java Jpms Flink Java11 | N/A | beam_PostCommit_Java_Jpms_Flink_Java11.json |
 |
| PostCommit Java Jpms Spark Java11 | N/A | beam_PostCommit_Java_Jpms_Spark_Java11.json |
 |
| PostCommit Java Nexmark Dataflow | N/A | beam_PostCommit_Java_Nexmark_Dataflow.json |
 |
| PostCommit Java Nexmark Dataflow V2 | N/A | beam_PostCommit_Java_Nexmark_Dataflow_V2.json |
 |
| PostCommit Java Nexmark Dataflow V2 Java | ['11','17'] | beam_PostCommit_Java_Nexmark_Dataflow_V2_Java.json |
 |
| PostCommit Java Nexmark Direct | N/A | beam_PostCommit_Java_Nexmark_Direct.json |
 |
| PostCommit Java Nexmark Flink | N/A | beam_PostCommit_Java_Nexmark_Flink.json |
 |
| PostCommit Java Nexmark Spark | N/A | beam_PostCommit_Java_Nexmark_Spark.json |
 |
| PostCommit Java PVR Flink Streaming | N/A | beam_PostCommit_Java_PVR_Flink_Streaming.json |
 |
| PostCommit Java PVR Samza | N/A | beam_PostCommit_Java_PVR_Samza.json |
 |
| PostCommit Java SingleStoreIO IT | N/A | beam_PostCommit_Java_SingleStoreIO_IT.json |
 |
| PostCommit Java PVR Spark3 Streaming | N/A | beam_PostCommit_Java_PVR_Spark3_Streaming.json |
 |
| PostCommit Java PVR Spark Batch | N/A | beam_PostCommit_Java_PVR_Spark_Batch.json |
 |
| PostCommit Java Tpcds Dataflow | N/A | beam_PostCommit_Java_Tpcds_Dataflow.json |
 |
| PostCommit Java Tpcds Flink | N/A | beam_PostCommit_Java_Tpcds_Flink.json |
 |
| PostCommit Java Tpcds Spark | N/A | beam_PostCommit_Java_Tpcds_Spark.json |
 |
| PostCommit Java ValidatesRunner Dataflow JavaVersions | ['8','21'] | beam_PostCommit_Java_ValidatesRunner_Dataflow_JavaVersions.json |
 |
| PostCommit Java ValidatesRunner Dataflow Streaming | N/A | beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming.json |
 |
| PostCommit Java ValidatesRunner Dataflow V2 Streaming | N/A | beam_PostCommit_Java_ValidatesRunner_Dataflow_V2_Streaming.json |
 |
| PostCommit Java ValidatesRunner Dataflow V2 | N/A | beam_PostCommit_Java_ValidatesRunner_Dataflow_V2.json |
 |
| PostCommit Java ValidatesRunner Dataflow | N/A | beam_PostCommit_Java_ValidatesRunner_Dataflow.json |
 |
| PostCommit Java ValidatesRunner Direct JavaVersions | ['8','21'] | beam_PostCommit_Java_ValidatesRunner_Direct_JavaVersions.json |
 |
| PostCommit Java ValidatesRunner Direct | N/A | beam_PostCommit_Java_ValidatesRunner_Direct.json |
 |
| PostCommit Java ValidatesRunner Flink Java8 | N/A | beam_PostCommit_Java_ValidatesRunner_Flink_Java8.json |
 |
| PostCommit Java ValidatesRunner Flink | N/A | beam_PostCommit_Java_ValidatesRunner_Flink.json |
 |
| PostCommit Java ValidatesRunner Samza | N/A | beam_PostCommit_Java_ValidatesRunner_Samza.json |
 |
| PostCommit Java ValidatesRunner Spark Java8 | N/A | beam_PostCommit_Java_ValidatesRunner_Spark_Java8.json |
 |
| PostCommit Java ValidatesRunner Spark | N/A | beam_PostCommit_Java_ValidatesRunner_Spark.json |
 |
| PostCommit Java ValidatesRunner SparkStructuredStreaming | N/A | beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.json |
 |
| PostCommit Java ValidatesRunner Twister2 | N/A | beam_PostCommit_Java_ValidatesRunner_Twister2.json |
 |
| PostCommit Java ValidatesRunner ULR | N/A | beam_PostCommit_Java_ValidatesRunner_ULR.json |
 |
| PostCommit Java | N/A | beam_PostCommit_Java.json |
 |
| PostCommit Javadoc | N/A | beam_PostCommit_Javadoc.json |
 |
| PostCommit PortableJar Flink | N/A | beam_PostCommit_PortableJar_Flink.json |
 |
| PostCommit PortableJar Spark | N/A | beam_PostCommit_PortableJar_Spark.json |
 |
| PostCommit Python | ['3.8','3.9','3.10','3.11'] | beam_PostCommit_Python.json |
 |
| PostCommit Python Arm | ['3.8','3.9','3.10','3.11'] | beam_PostCommit_Python_Arm.json |
 |
| PostCommit Python Dependency | N/A | beam_PostCommit_Python_Dependency.json |
 |
| PostCommit Python Examples Dataflow | N/A | beam_PostCommit_Python_Examples_Dataflow.json |
 |
| PostCommit Python Examples Direct | ['3.8','3.9','3.10','3.11'] | beam_PostCommit_Python_Examples_Direct.json |
 |
| PostCommit Python Examples Flink | ['3.8','3.11'] | beam_PostCommit_Python_Examples_Flink.json |
 |
| PostCommit Python Examples Spark | ['3.8','3.11'] | beam_PostCommit_Python_Examples_Spark.json |
 |
| PostCommit Python MongoDBIO IT | N/A | beam_PostCommit_Python_MongoDBIO_IT.json |
 |
| PostCommit Python Nexmark Direct | N/A | beam_PostCommit_Python_Nexmark_Direct.json |
 |
| PostCommit Python ValidatesContainer Dataflow | ['3.8','3.9','3.10','3.11'] | beam_PostCommit_Python_ValidatesContainer_Dataflow.json |
 |
| PostCommit Python ValidatesContainer Dataflow With RC | ['3.8','3.9','3.10','3.11'] | beam_PostCommit_Python_ValidatesContainer_Dataflow_With_RC.json |
 |
| PostCommit Python ValidatesRunner Dataflow | ['3.8','3.11'] | beam_PostCommit_Python_ValidatesRunner_Dataflow.json |
 |
| PostCommit Python ValidatesRunner Flink | ['3.8','3.11'] | beam_PostCommit_Python_ValidatesRunner_Flink.json |
 |
| PostCommit Python ValidatesRunner Samza | ['3.8','3.11'] | beam_PostCommit_Python_ValidatesRunner_Samza.json |
 |
| PostCommit Python ValidatesRunner Spark | ['3.8','3.9','3.11'] | beam_PostCommit_Python_ValidatesRunner_Spark.json |
 |
| PostCommit Python Xlang Gcp Dataflow | N/A | beam_PostCommit_Python_Xlang_Gcp_Dataflow.json |
 |
| PostCommit Python Xlang Gcp Direct | N/A | beam_PostCommit_Python_Xlang_Gcp_Direct.json |
 |
| PostCommit Python Xlang IO Dataflow | N/A | beam_PostCommit_Python_Xlang_IO_Dataflow.json |
 |
| PostCommit SQL | N/A | beam_PostCommit_SQL.json |
 |
| PostCommit TransformService Direct | N/A | beam_PostCommit_TransformService_Direct.json |
 |
| PostCommit Website Test | N/A | beam_PostCommit_Website_Test.json |
 |
| PostCommit XVR GoUsingJava Dataflow | N/A | beam_PostCommit_XVR_GoUsingJava_Dataflow.json |
 |
| PostCommit XVR Direct | N/A | beam_PostCommit_XVR_Direct.json |
 |
| PostCommit XVR Flink | N/A | beam_PostCommit_XVR_Flink.json |
 |
| PostCommit XVR JavaUsingPython Dataflow | N/A | beam_PostCommit_XVR_JavaUsingPython_Dataflow.json |
 |
| PostCommit XVR PythonUsingJava Dataflow | N/A | beam_PostCommit_XVR_PythonUsingJava_Dataflow.json |
 |
| PostCommit XVR PythonUsingJavaSQL Dataflow | N/A | beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow.json |
 |
| PostCommit XVR Samza | N/A | beam_PostCommit_XVR_Samza.json |
 |
| PostCommit XVR Spark3 | N/A | beam_PostCommit_XVR_Spark3.json |
 |
| Python Validates Container Dataflow ARM | ['3.8','3.9','3.10','3.11'] | beam_Python_ValidatesContainer_Dataflow_ARM.json |
 |
| Workflow name | Matrix | Cron Status |
|---|---|---|
| Cancel Stale Dataflow Jobs | N/A |  |
| Clean Up GCP Resources | N/A |  |
| Clean Up Prebuilt SDK Images | N/A |  |
| Cleanup Dataproc Resources | N/A |  |
| Community Metrics Prober | N/A |  |
| PostRelease Nightly Snapshot | N/A |  |
| Playground CI Nightly | ["java", "go", "python"] |  |
| Publish Beam SDK Snapshots | N/A |  |
| Publish BeamMetrics | N/A |  |
| Publish Docker Snapshots | N/A |  |
| Publish Website | N/A |  |
| Release Nightly Snapshot | N/A |  |
| Release Nightly Snapshot Python | N/A |  |
| Rotate IO-Datastores Cluster Credentials | N/A |  |
| Rotate Metrics Cluster Credentials | N/A |  |