Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test metrics are no longer pushed to Comet.ML (and perhaps others) #1460

Closed
alsrgv opened this issue Apr 11, 2020 · 6 comments · Fixed by #1459
Closed

Test metrics are no longer pushed to Comet.ML (and perhaps others) #1460

alsrgv opened this issue Apr 11, 2020 · 6 comments · Fixed by #1459
Labels
bug Something isn't working help wanted Open to be worked on
Milestone

Comments

@alsrgv
Copy link
Contributor

alsrgv commented Apr 11, 2020

🐛 Bug

PyTorch Lightning 0.7.2 used to publish test metrics to Comet.ML. Commit ddbf7de has broken this functionality.

To Reproduce

Steps to reproduce the behavior:

Run fast-run of training and observe test metrics not being submitted to Comet.ML (and possibly other logging destinations).

Environment

cuda:
        GPU:
                Tesla T4
        available:           True
        version:             10.1
packages:
        numpy:               1.17.2
        pyTorch_debug:       False
        pyTorch_version:     1.4.0
        pytorch-lightning:   0.7.4-dev
        tensorboard:         2.2.0
        tqdm:                4.45.0
system:
        OS:                  Linux
        architecture:
                64bit

        processor:           x86_64
        python:              3.6.8
        version:             #69-Ubuntu SMP Thu Mar 26 02:17:29 UTC 2020

cc @alexeykarnachev

@alsrgv alsrgv added bug Something isn't working help wanted Open to be worked on labels Apr 11, 2020
@github-actions
Copy link
Contributor

Hi! thanks for your contribution!, great first issue!

@williamFalcon
Copy link
Contributor

@PyTorchLightning/core-contributors or @alsrgv mind submitting a PR? good catch!

@alsrgv
Copy link
Contributor Author

alsrgv commented Apr 11, 2020

Happy to, but I could use some pointers into what may be broken. Does logging use aggregation with flush in the end, and that flush is somehow not called for the test pass? @alexeykarnachev, any ideas?

@Borda
Copy link
Member

Borda commented Apr 12, 2020

Shall be fixed in #1459

@Borda Borda mentioned this issue Apr 12, 2020
5 tasks
@Borda Borda added this to the 0.7.4 milestone Apr 12, 2020
@Borda Borda linked a pull request Apr 12, 2020 that will close this issue
5 tasks
@alexeykarnachev
Copy link
Contributor

Sorry, guys, totally missed the messages.
@Borda , is anything required from my end?

@Borda
Copy link
Member

Borda commented Apr 14, 2020

I think it is fine, just if you have an idea why the Github Actions fails/hangs...
https://github.com/PyTorchLightning/pytorch-lightning/pull/1459/checks?check_run_id=584135478

@Borda Borda modified the milestones: 0.7.4, v0.7.x Apr 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Open to be worked on
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants