Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable triggering postcommit via specific file #29549

Merged
merged 3 commits into from
Nov 30, 2023

Conversation

Abacn
Copy link
Contributor

@Abacn Abacn commented Nov 28, 2023

Workaround #28909

Enable to trigger a PostCommit from pull request by touching a file named "<workflow_name>.json" (/json to get around with RAT PreCommit), as a simple workaround of #28909.

In this way the last piece of gap (comment trigger) is partially fixed and we can shutdown all Jenkins jobs.

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@github-actions github-actions bot added the build label Nov 28, 2023
@@ -271,7 +271,6 @@ Please note that jobs with matrix need to have matrix element in the comment. Ex

| Workflow name | Matrix | Trigger Phrase | Cron Status |
|:-------------:|:------:|:--------------:|:-----------:|
| [ PostCommit BeamMetrics Publish ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_BeamMetrics_Publish.yml) | N/A |`Run Beam Metrics Deployment`| [![.github/workflows/beam_PostCommit_BeamMetrics_Publish.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_BeamMetrics_Publish.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_BeamMetrics_Publish.yml?query=event%3Aschedule)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BeamMetrics Publish and Website Publish are not really "postcommit" or test suites. It deploys the Beam website or deploys the community metrics dashboard. They should not have comment trigger (which could be used to attack beam website). So renamed them and move to "other" category.

Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @damccorm for label build.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Copy link
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on board with this approach, thanks

@@ -271,7 +271,6 @@ Please note that jobs with matrix need to have matrix element in the comment. Ex

| Workflow name | Matrix | Trigger Phrase | Cron Status |
|:-------------:|:------:|:--------------:|:-----------:|
| [ PostCommit BeamMetrics Publish ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_BeamMetrics_Publish.yml) | N/A |`Run Beam Metrics Deployment`| [![.github/workflows/beam_PostCommit_BeamMetrics_Publish.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_BeamMetrics_Publish.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_BeamMetrics_Publish.yml?query=event%3Aschedule)
| [ PostCommit Go ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Go.yml) | N/A |`Run Go PostCommit`| [![.github/workflows/beam_PostCommit_Go.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Go.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Go.yml?query=event%3Aschedule) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to also update all the postcommits in this file to not have trigger phrases listed. Ideally we'd list the file to modify instead.

We should also modify the part that talks about trigger phrases at the top and explain this approach.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated README.md replacing trigger phrase->trigger file, added instructions on how to manually trigger precommit and postcommits

@@ -19,7 +19,7 @@ on:
schedule:
- cron: '0 0 * * 0'
pull_request_target:
paths: ['release/trigger_all_tests.json']
paths: ['release/trigger_all_tests.json', 'beam_Java_JMH.json']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put these in a subfolder? Maybe .github/trigger_files? That way if they accidentally get checked in it doesn't cause issues.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our guidance could even be to check them in, it does no harm.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added prefix .github/trigger_files

Copy link
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@@ -192,6 +192,8 @@ Please note that jobs with matrix need to have matrix element in the comment. Ex

### PreCommit Jobs

PreCommit Jobs run in a schedule and also get triggered in a PR if relevant sources has changed. To manually trigger certain PreCommit job, comment with the Trigger Phrase (listed below) in the PR.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add something for postcommit jobs here with instructions on how to find the magic file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostCommit instructions is in PostCommit subsection: https://github.com/apache/beam/blob/6efa27c6e08513ae5b3667d9efcaa6c660525cfc/.github/workflows/README.md#postcommit-jobs

yeah its not quite easy to find given the long README file. Will also mention it in the #comment-triggering-support section

@damccorm
Copy link
Contributor

Oh, we should probably do the same for benchmarks as well

@Abacn
Copy link
Contributor Author

Abacn commented Nov 30, 2023

Oh, we should probably do the same for benchmarks as well

The hope is we can still have the comment trigger in the near future, which is more convenient for developers and reviewers (reviewer can also run the test by comment), as @volatilemolotov working on GitHub App approach. If decided this being long time solution then we can add it to load tests and performance tests

@Abacn Abacn merged commit c6c7086 into apache:master Nov 30, 2023
@Abacn Abacn deleted the triggerpostcommit branch January 11, 2024 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants