-
Notifications
You must be signed in to change notification settings - Fork 36
fix ci: account for huggingface transformers changes #781
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
3717370
to
36e0d31
Compare
b0cb886
to
522bbd8
Compare
ff721e7
to
ad04ca4
Compare
ad04ca4
to
6bef79e
Compare
My current theory is that a second instance of |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #781 +/- ##
==========================================
+ Coverage 88.79% 95.55% +6.75%
==========================================
Files 53 55 +2
Lines 3357 3532 +175
Branches 294 314 +20
==========================================
+ Hits 2981 3375 +394
+ Misses 337 110 -227
- Partials 39 47 +8 β View full report in Codecov by Sentry. |
4ed3107
to
e337f08
Compare
e337f08
to
27e38cb
Compare
@@ -101,6 +101,7 @@ def args(): | |||
evaluation_strategy="epoch", | |||
num_train_epochs=2, | |||
save_strategy="epoch", | |||
report_to="none", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Docs are here. Having this set to the default of "all"
seems to create a second instance of the DVCLiveCallback
which causes duplicate entries to be logged in all of the metric files.
@dberenbaum is this something we need to be aware of for #740?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
total_flos was also added as a metric before report_to
is set to "none"
(original failure)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch @mattseddon! Looks like it's causeg by testing the old callback that was part of dvclive along with transformers>=4.36.0
, which auto reports using the new callback that's part of transformers. I will take a look for #740 and ask you to review.
β I have followed the Contributing to DVCLive guide.
π If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.
Thank you for the contribution - we'll try to review it as soon as possible. π
This PR fixes the
test_huggingface_integration
test. Check the inline comment for an explanation.Closes #768