Importing nicegui affects mp.set_start_method, throws context has already been set

[EricCousineau-TRI]

Description

Using nicegui == 1.4.34, CPython 3.10.12, on Ubuntu 22.04.

The following works fine:

$ python -c 'import multiprocessing as mp; mp.set_start_method("spawn")'

The following, however, fails:

$ python -c 'import nicegui; import multiprocessing as mp; mp.set_start_method("spawn")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.10/multiprocessing/context.py", line 247, in set_start_method
    raise RuntimeError('context has already been set')
RuntimeError: context has already been set

[rodja]

I currently have not the time to dig deeper but it seems to be related to the auto-reloading mechanism of uvicorn. I could reproduce the same behaviour without NiceGUI:

import multiprocessing as mp
import uvicorn

mp.set_start_method("spawn")

if __name__ == "__main__":
    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)

If you set ui.run(reload=False) the start method can be set.

[falkoschindler]

We don't think this is an issue with NiceGUI.
If you wish to be able to change the start method in a NiceGUI app (without force=True), feel free to create a feature request or even a pull request. Thanks!

[EricCousineau-TRI]

Thanks for the feedback!

The underlying issue is importing either nicegui (or something it imports) creates a side effect where we can neither call multiprocessing.set_start_method("spawn").

Note that the error above happens before any code is actually "run" -- I am not calling app.run(...) or uvicorn.run(...).
Perhaps the issue is not with nicegui, but something in the import chain is causing a functional side effect at import time, which ideally should happen when code is explicitly called (e.g. via the main() function).

For us, we simply worked around it by deferring the import of any nicegui components to our main() function. This works, but is a little "dirty".

Does this make sense?

[EricCousineau-TRI]

Also note that the above still fails for nicegui == 2.0.1.

It does not fail for uvicorn == 0.30.6:

$ python -c 'import multiprocessing as mp; import uvicorn; mp.set_start_method("spawn")'
# No error.

[EricCousineau-TRI]

If I import nicegui, then collect all the modules registered in sys.modules, and import them individually, it seems like nicegui is the only module that, when imported, causes an issue:
EricCousineau-TRI/repro@3d7cd94

So this seems like it would indicate an issue somewhere in nicegui.

[rodja]

So this seems like it would indicate an issue somewhere in nicegui.

Sure. I can reproduce it with NiceGUI. But as said, it can be also be reproduced with uvicorn alone. So the underlying problem is "upstream". Maybe there is a workaround we can implement in NiceGUI, but I doubt that.

Note that the error above happens before any code is actually "run"

At first glance it looks that way. But it's important to understand the full startup process: With ui.run(reload=True) (which is the default), the main module is executed twice. Uvicorns main first process spawns a subprocess and launches the actual server which is killed and relaunched on file change" as we wrote in our FAQ.

[davetapley]

@rodja FYI it's no long reproducible upstream in uvicorn:

$ pip show uvicorn | grep Version
Version: 0.34.0
$ python -c 'import multiprocessing as mp; import uvicorn; mp.set_start_method("spawn")'

$ pip show nicegui | grep Version
Version: 2.14.0
$ python -c 'import multiprocessing as mp; import nicegui; mp.set_start_method("spawn")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/python/3.12.2/lib/python3.12/multiprocessing/context.py", line 247, in set_start_method
    raise RuntimeError('context has already been set')

[davetapley]

Once #4613 is merged this will work with:

set_start_method("spawn", force=True)
ui.run(title='My App', port=8081, reload=False, require_main_process=False)