Skip to content

[fbgemm_gpu] Add Scripts for Generating Release Reports #3676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

q10
Copy link
Contributor

@q10 q10 commented Feb 11, 2025

  • Add scripts for generating release reports

Copy link

netlify bot commented Feb 11, 2025

Deploy Preview for pytorch-fbgemm-docs failed.

Name Link
🔨 Latest commit 74ab596
🔍 Latest deploy log https://app.netlify.com/sites/pytorch-fbgemm-docs/deploys/67f4825ce345df000875deec

@q10 q10 force-pushed the bm/scrape-commits branch 4 times, most recently from 0bae084 to 06e7f6b Compare April 7, 2025 18:10
@q10 q10 changed the title [wip] commits scraper [fbgemm_gpu] Releases Reporting Scripts Apr 7, 2025
@q10 q10 changed the title [fbgemm_gpu] Releases Reporting Scripts [fbgemm_gpu] Add Scripts for Generating Release Reports Apr 7, 2025
@facebook-github-bot
Copy link
Contributor

@q10 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

q10 added a commit to q10/FBGEMM that referenced this pull request Apr 7, 2025
Summary:
- Add scripts for generating release reports


Differential Revision: D72581171

Pulled By: q10
@q10 q10 force-pushed the bm/scrape-commits branch from 06e7f6b to 8239b1e Compare April 7, 2025 18:23
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D72581171

q10 added a commit to q10/FBGEMM that referenced this pull request Apr 7, 2025
Summary:
X-link: facebookresearch/FBGEMM#1022

- Add scripts for generating release reports


Differential Revision: D72581171

Pulled By: q10
@q10 q10 force-pushed the bm/scrape-commits branch from 8239b1e to b33f5d1 Compare April 7, 2025 18:28
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D72581171

q10 added a commit to q10/FBGEMM that referenced this pull request Apr 7, 2025
Summary:
X-link: facebookresearch/FBGEMM#1022

- Add scripts for generating release reports


Differential Revision: D72581171

Pulled By: q10
@q10 q10 force-pushed the bm/scrape-commits branch from b33f5d1 to 5ca16af Compare April 7, 2025 18:31
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D72581171

q10 added a commit to q10/FBGEMM that referenced this pull request Apr 7, 2025
Summary:
Add scripts for generating release reports.

The previous interation of the scraper script was based on a flawed understanding of the GItHub PR API.  Initially, I thought it was possible to grep the API for merged PRs in reverse sorted order, but it turns out that PR timestamps can be updated after merge, and PR information is based on open/close state of PR, not merge state.  

So it is much better to first fetch all parent commits for a given branch/tag/ref from a start timetamp, then for each commit, search the API for its associated PR, and then pull the PR information.  Even though this invokes more API calls and is slower, it is more accurate.

Additionally, the code has been rewritten to support more operations in the future.  For example, in this diff is a new command that fetches unlabeled PRs, which is useful for us as we construct a release report.


Differential Revision: D72581171

Pulled By: q10
@q10 q10 force-pushed the bm/scrape-commits branch from 5ca16af to ecca356 Compare April 7, 2025 19:48
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D72581171

q10 added a commit to q10/FBGEMM that referenced this pull request Apr 8, 2025
Summary:
Add scripts for generating release reports.

The previous interation of the scraper script was based on a flawed understanding of the GItHub PR API.  Initially, I thought it was possible to grep the API for merged PRs in reverse sorted order, but it turns out that PR timestamps can be updated after merge, and PR information is based on open/close state of PR, not merge state.  

So it is much better to first fetch all parent commits for a given branch/tag/ref from a start timetamp, then for each commit, search the API for its associated PR, and then pull the PR information.  Even though this invokes more API calls and is slower, it is more accurate.

Additionally, the code has been rewritten to support more operations in the future.  For example, in this diff is a new command that fetches unlabeled PRs, which is useful for us as we construct a release report.

Example releases report: P1778912161
Example report of unlabeled PRs: P1778913261


Test Plan:
Imported from GitHub, without a `Test Plan:` line.

Example:

```
python scrape.py release-report --ref v1.1.2-rc1 --since 2025-04-01
```

Differential Revision: D72581171

Pulled By: q10
@q10 q10 force-pushed the bm/scrape-commits branch from ecca356 to bd386e3 Compare April 8, 2025 01:29
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D72581171

Summary:
X-link: facebookresearch/FBGEMM#1022

Add scripts for generating release reports.

The previous interation of the scraper script was based on a flawed understanding of the GItHub PR API.  Initially, I thought it was possible to grep the API for merged PRs in reverse sorted order, but it turns out that PR timestamps can be updated after merge, and PR information is based on open/close state of PR, not merge state.  

So it is much better to first fetch all parent commits for a given branch/tag/ref from a start timetamp, then for each commit, search the API for its associated PR, and then pull the PR information.  Even though this invokes more API calls and is slower, it is more accurate.

Additionally, the code has been rewritten to support more operations in the future.  For example, in this diff is a new command that fetches unlabeled PRs, which is useful for us as we construct a release report.

Example releases report: P1778912161
Example report of unlabeled PRs: P1778913261


Test Plan:
Imported from GitHub, without a `Test Plan:` line.

Example:

```
python scrape.py release-report --ref v1.1.2-rc1 --since 2025-04-01
```

Differential Revision: D72581171

Pulled By: q10
@q10 q10 force-pushed the bm/scrape-commits branch from bd386e3 to 74ab596 Compare April 8, 2025 01:56
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D72581171

@facebook-github-bot
Copy link
Contributor

@q10 merged this pull request in b38098f.

q10 added a commit to q10/FBGEMM that referenced this pull request Apr 10, 2025
Summary:
Pull Request resolved: facebookresearch/FBGEMM#1022

Add scripts for generating release reports.

The previous interation of the scraper script was based on a flawed understanding of the GItHub PR API.  Initially, I thought it was possible to grep the API for merged PRs in reverse sorted order, but it turns out that PR timestamps can be updated after merge, and PR information is based on open/close state of PR, not merge state.

So it is much better to first fetch all parent commits for a given branch/tag/ref from a start timetamp, then for each commit, search the API for its associated PR, and then pull the PR information.  Even though this invokes more API calls and is slower, it is more accurate.

Additionally, the code has been rewritten to support more operations in the future.  For example, in this diff is a new command that fetches unlabeled PRs, which is useful for us as we construct a release report.

Example releases report: P1778912161
Example report of unlabeled PRs: P1778913261

X-link: pytorch#3676

Test Plan:
Imported from GitHub, without a `Test Plan:` line.

Example:

```
python scrape.py release-report --ref v1.1.2-rc1 --since 2025-04-01
```

Reviewed By: spcyppt

Differential Revision: D72581171

Pulled By: q10

fbshipit-source-id: 88e998a1dcc98078526dcb7e1b2d566b467080ea
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants