Skip to content

Commit fe35eb3

Browse files
committed
make iterating over sys.modules threadsafe
Despite creating a copy through list(sys.modules.items()) there is a possible race condition if another thread is adding to sys.modules File "x/lib/python3.7/pickle.py", line 774, in save_tuple save(element) File "x/lib/python3.7/pickle.py", line 549, in save self.save_reduce(obj=obj, *rv) File "x/lib/python3.7/pickle.py", line 637, in save_reduce save(func) File "x/lib/python3.7/pickle.py", line 518, in save self.save_global(obj) File "x/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 876, in save_global elif not _is_global(obj, name=name): File "x/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 174, in _is_global module_name = _whichmodule(obj, name) File "x/lib/python3.7/site-packages/cloudpickle/cloudpickle.py", line 156, in _whichmodule for module_name, module in list(sys.modules.items()): RuntimeError: dictionary changed size during iteration
1 parent 8cf9ec4 commit fe35eb3

File tree

2 files changed

+8
-3
lines changed

2 files changed

+8
-3
lines changed

Diff for: CHANGES.md

+4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
1.2.3
22
=====
33

4+
- Fix a bug when a thread imports a module while cloudpickle iterates
5+
over the module list
6+
([PR #322](https://github.com/cloudpipe/cloudpickle/pull/322)).
7+
48
1.2.2
59
=====
610

Diff for: cloudpickle/cloudpickle.py

+4-3
Original file line numberDiff line numberDiff line change
@@ -151,9 +151,10 @@ def _whichmodule(obj, name):
151151
module_name = getattr(obj, '__module__', None)
152152
if module_name is not None:
153153
return module_name
154-
# Protect the iteration by using a list copy of sys.modules against dynamic
155-
# modules that trigger imports of other modules upon calls to getattr.
156-
for module_name, module in list(sys.modules.items()):
154+
# Protect the iteration by using a copy of sys.modules against dynamic
155+
# modules that trigger imports of other modules upon calls to getattr or
156+
# other threads importing at the same time.
157+
for module_name, module in sys.modules.copy().items():
157158
if module_name == '__main__' or module is None:
158159
continue
159160
try:

0 commit comments

Comments
 (0)