Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixed extra dataloader bug #1196

Merged
merged 14 commits into from
Apr 2, 2020
Merged

fixed extra dataloader bug #1196

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
0b6832f
fixed extra dataloader bug
TevenLeScao Mar 19, 2020
c805f94
Update pytorch_lightning/trainer/training_loop.py
TevenLeScao Mar 20, 2020
02349a9
updated CHANGELOG
TevenLeScao Mar 20, 2020
79b1b3a
Merge remote-tracking branch 'origin/extra_dataloader_bug-fix' into e…
TevenLeScao Mar 20, 2020
d8241a8
Small non-repetition change
TevenLeScao Mar 20, 2020
6b41ed2
Update CHANGELOG.md
Borda Mar 20, 2020
9a45c12
Merge branch 'master' into extra_dataloader_bug-fix
TevenLeScao Mar 23, 2020
00f3689
changed argument name to reload_train_dataloader_every_epoch
TevenLeScao Mar 30, 2020
76ed916
Merge remote-tracking branch 'origin/extra_dataloader_bug-fix' into e…
TevenLeScao Mar 30, 2020
f52d30a
fixed doc underline too short
TevenLeScao Mar 30, 2020
8304bcc
Merge branch 'master' into extra_dataloader_bug-fix
TevenLeScao Mar 30, 2020
899ab4f
reverted to `reload_dataloaders_every_epoch`
TevenLeScao Mar 30, 2020
53d3b14
fixed val and test reloading
TevenLeScao Mar 31, 2020
4b5e599
fixed val and test reloading
TevenLeScao Mar 31, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).

- Fixed bug related to type cheking of `ReduceLROnPlateau` lr schedulers([#1114](https://github.com/PyTorchLightning/pytorch-lightning/issues/1114))
- Fixed a bug to ensure lightning checkpoints to be backward compatible ([#1132](https://github.com/PyTorchLightning/pytorch-lightning/pull/1132))
- Fixed a bug that created an extra dataloader with active `reload_dataloaders_every_epoch` ([#1181](https://github.com/PyTorchLightning/pytorch-lightning/issues/1181)
- Fixed all warnings and errors in the docs build process ([#1191](https://github.com/PyTorchLightning/pytorch-lightning/pull/1191))

## [0.7.1] - 2020-03-07
Expand Down
11 changes: 6 additions & 5 deletions pytorch_lightning/trainer/training_loop.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,9 @@ def train(self):
model = self.get_model()

# load data
self.reset_train_dataloader(model)
# if reload_dataloaders_every_epoch, this is moved to the epoch loop
if not self.reload_dataloaders_every_epoch:
self.reset_train_dataloader(model)
self.reset_val_dataloader(model)

# Train start events
Expand All @@ -304,6 +306,9 @@ def train(self):
try:
# run all epochs
for epoch in range(self.current_epoch, self.max_epochs):
# reset train dataloader
if self.reload_dataloaders_every_epoch:
self.reset_train_dataloader(model)
# set seed for distributed sampler (enables shuffling for each epoch)
if self.use_ddp \
and hasattr(self.train_dataloader.sampler, 'set_epoch'):
Expand Down Expand Up @@ -386,10 +391,6 @@ def run_training_epoch(self):
if self.is_function_implemented('on_epoch_start'):
self.get_model().on_epoch_start()

# reset train dataloader
if self.reload_dataloaders_every_epoch:
self.reset_train_dataloader(self.get_model())

# track local dataloader so TPU can wrap each epoch
train_dataloader = self.train_dataloader

Expand Down