Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

multiprocessing cpu only training #222

Closed
artemru opened this issue Sep 12, 2019 · 7 comments
Closed

multiprocessing cpu only training #222

artemru opened this issue Sep 12, 2019 · 7 comments
Assignees
@mtnwni
Labels
feature Is an improvement or enhancement good first issue Good for newcomers

Comments

@artemru
Copy link

artemru commented Sep 12, 2019

I would like to know if it's possible to train a model with multiprocess parallelism (no GPU available) using Lightning (sync analogue of https://pytorch.org/docs/stable/notes/multiprocessing.html#hogwild) ? After a quick glance, I've the impression that in Trainer all available options for parallelism are GPU based (if I'm not mistaken torch.DPD supports multiproc CPU-only training).
@artemru artemru added the question Further information is requested label Sep 12, 2019
@williamFalcon
Copy link
Contributor

good question. we only support distributed GPU but would welcome a PR to support multi-CPU options.

@neggert
Copy link
Contributor

neggert commented Sep 12, 2019

Hah! I was just looking for this. If nothing else, it would make testing DDP much easier.

@stale
Copy link

stale bot commented Feb 22, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the won't fix This will not be worked on label Feb 22, 2020
@Borda
Copy link
Member

Borda commented Feb 22, 2020

@artemru @neggert are you interested in sending a PR? 🤖

@stale stale bot removed the won't fix This will not be worked on label Feb 22, 2020
@Borda Borda added feature Is an improvement or enhancement good first issue Good for newcomers labels Feb 22, 2020
@mtnwni
Copy link
Contributor

mtnwni commented Mar 4, 2020

I would like to take this up.

@Borda
Copy link
Member

Borda commented Mar 4, 2020

Cool! Thx @skepticleo

@awaelchli
Copy link
Contributor

awaelchli commented Apr 25, 2020 •
edited
Loading

Looks like this was recently added in #1158 :)

@Borda Borda closed this as completed Apr 25, 2020
@Borda Borda removed the question Further information is requested label Dec 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mtnwni mtnwni

Labels
feature Is an improvement or enhancement good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@artemru @neggert @mtnwni @williamFalcon @awaelchli @Borda