Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Auto-scaling batch-size not compatible with half precision training #3025

Closed
iantimmis opened this issue Aug 17, 2020 · 2 comments · Fixed by #3045
Closed

Auto-scaling batch-size not compatible with half precision training #3025

iantimmis opened this issue Aug 17, 2020 · 2 comments · Fixed by #3045
Assignees
@awaelchli
Labels
bug Something isn't working help wanted Open to be worked on priority: 0 High priority task
Milestone
pl:0.9

Comments

@iantimmis
Copy link

Using precision=16 and auto_scale_batch_size=True yields 'NoneType' object has no attribute 'state_dict' error

Expected behavior

Largest batch size should be found when using 16 bit precision

Environment

  • PyTorch Version (e.g., 1.0): 1.6
  • OS (e.g., Linux): Windows 10
  • How you installed PyTorch (conda, pip, source): conda
  • Python version: 3.8
  • CUDA/cuDNN version: 10/7
  • GPU models and configuration: 2080ti
@iantimmis iantimmis added bug Something isn't working help wanted Open to be worked on labels Aug 17, 2020
@github-actions
Copy link
Contributor

Hi! thanks for your contribution!, great first issue!

@edenlightning edenlightning added the priority: 0 High priority task label Aug 18, 2020
@Borda
Copy link
Member

Borda commented Aug 18, 2020

it seems that loading has some issue
@iantimmis are interested in sending a PR?

@edenlightning edenlightning added this to the 0.9.0 milestone Aug 18, 2020
@awaelchli awaelchli self-assigned this Aug 19, 2020
@awaelchli awaelchli mentioned this issue Aug 19, 2020
Merged
7 tasks
@mergify mergify bot closed this as completed in #3045 Aug 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@awaelchli awaelchli

Labels
bug Something isn't working help wanted Open to be worked on priority: 0 High priority task
Projects
None yet
Milestone
pl:0.9
Development

Successfully merging a pull request may close this issue.

fix auto scale batch size not working with precision=16 Lightning-AI/pytorch-lightning
4 participants
@awaelchli @Borda @iantimmis @edenlightning