Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

V0.9.22 bug: error in running finalizer: UndefVarError(var=:PYNULL) #543

Closed
ppalmes opened this issue Aug 21, 2024 · 5 comments · Fixed by #544
Closed

V0.9.22 bug: error in running finalizer: UndefVarError(var=:PYNULL) #543

ppalmes opened this issue Aug 21, 2024 · 5 comments · Fixed by #544
Labels
bug Something isn't working

Comments

@ppalmes
Copy link

ppalmes commented Aug 21, 2024

Affects: PythonCall

Describe the bug

error in running finalizer: UndefVarError(var=:PYNULL)
ijl_undefined_var_error at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/rtutils.c:134
ijl_get_binding_or_error at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/module.c:471
enqueue_all at /home/ppalmes/.julia/packages/PythonCall/flx5V/src/GC/GC.jl:106
pyobjectarray_finalizer at /home/ppalmes/.julia/packages/PythonCall/flx5V/src/JlWrap/objectarray.jl:30
unknown function (ip: 0x7fbc2d9fcca5)

Your system

julia> Pkg.status()
Project AutoMLPipeline v0.4.4
Status `~/phome/julia/AutoMLPipeline.jl/Project.toml`
  [e3c3008a] AMLPipelineBase v0.1.17
  [992eb4ea] CondaPkg v0.2.23
  [a93c6f00] DataFrames v1.6.1
  [6099a3de] PythonCall v0.9.22
  [9a3f8284] Random
  [8dfed614] Test

julia> CondaPkg.status()
CondaPkg Status /Users/ppalmes/phome/julia/AutoMLPipeline.jl/CondaPkg.toml
Environment
  /Users/ppalmes/phome/julia/AutoMLPipeline.jl/.CondaPkg/env
Packages
  python v3.12.5
  scikit-learn v1.5.1
Channels
  mkl
  scikit-learn

Additional context
Add any other context about the problem here.
@ppalmes ppalmes added the bug Something isn't working label Aug 21, 2024
@ppalmes
Copy link
Author

ppalmes commented Aug 21, 2024

I can validate that previous version (0.9.21) has no issue.

@felixcremer
Copy link

I get the same error when I run juliacall from python, but the computation finishes fine.

this is the full stack trace:

Details 

error in running finalizer: UndefVarError(var=:PYNULL)
ijl\_undefined\_var\_error at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/rtutils.c:134
ijl\_get\_binding\_or\_error at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/module.c:471
enqueue\_all at /home/b/b301089/.julia/packages/PythonCall/flx5V/src/GC/GC.jl:106
pyobjectarray\_finalizer at /home/b/b301089/.julia/packages/PythonCall/flx5V/src/JlWrap/objectarray.jl:30
unknown function (ip: 0x7ffb300fb477)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
run\_finalizer at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gc.c:318
jl\_gc\_run\_finalizers\_in\_list at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gc.c:410
run\_finalizers at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gc.c:454
run\_finalizers at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gc.c:435 \[inlined\]
ijl\_gc\_collect at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gc.c:3552
maybe\_collect at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gc.c:937 \[inlined\]
jl\_gc\_pool\_alloc\_inner at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gc.c:1300
ijl\_gc\_pool\_alloc at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gc.c:1348
innercode at /home/b/b301089/.julia/packages/YAXArrays/b5XBB/src/DAT/DAT.jl:1135
innerLoop at /home/b/b301089/.julia/packages/YAXArrays/b5XBB/src/DAT/DAT.jl:1179
unknown function (ip: 0x7ffb300cb447)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
jl\_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/julia.h:1982 \[inlined\]
do\_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/builtins.c:768
#100 at /home/b/b301089/.julia/packages/YAXArrays/b5XBB/src/DAT/DAT.jl:730
#46 at /home/b/b301089/.julia/packages/ProgressMeter/kVZZH/src/ProgressMeter.jl:1050
iterate at ./generator.jl:47 \[inlined\]
\_collect at ./array.jl:854
collect\_similar at ./array.jl:763
unknown function (ip: 0x7ffb300be8ab)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
map at ./abstractarray.jl:3285
unknown function (ip: 0x7ffb300bb29b)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
jl\_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/julia.h:1982 \[inlined\]
do\_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/builtins.c:768
macro expansion at /home/b/b301089/.julia/packages/ProgressMeter/kVZZH/src/ProgressMeter.jl:1049 \[inlined\]
macro expansion at ./task.jl:479 \[inlined\]
macro expansion at /home/b/b301089/.julia/packages/ProgressMeter/kVZZH/src/ProgressMeter.jl:1048 \[inlined\]
macro expansion at ./task.jl:479 \[inlined\]
#progress\_map#43 at /home/b/b301089/.julia/packages/ProgressMeter/kVZZH/src/ProgressMeter.jl:1041
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
jl\_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/julia.h:1982 \[inlined\]
do\_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/builtins.c:768
progress\_map at /home/b/b301089/.julia/packages/ProgressMeter/kVZZH/src/ProgressMeter.jl:1032
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
runLoop at /home/b/b301089/.julia/packages/YAXArrays/b5XBB/src/DAT/DAT.jl:725
unknown function (ip: 0x7fff8861e0bb)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
#mapCube#52 at /home/b/b301089/.julia/packages/YAXArrays/b5XBB/src/DAT/DAT.jl:498
unknown function (ip: 0x7fff885cc739)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
mapCube at /home/b/b301089/.julia/packages/YAXArrays/b5XBB/src/DAT/DAT.jl:418
unknown function (ip: 0x7fff885cb237)
#mapCube#37 at /home/b/b301089/.julia/packages/YAXArrays/b5XBB/src/DAT/DAT.jl:311 \[inlined\]
mapCube at /home/b/b301089/.julia/packages/YAXArrays/b5XBB/src/DAT/DAT.jl:311 \[inlined\]
#rqatrend#28 at /home/b/b301089/.julia/packages/RQADeforestation/e3X1u/src/analysis.jl:60 \[inlined\]
rqatrend at /home/b/b301089/.julia/packages/RQADeforestation/e3X1u/src/analysis.jl:58
unknown function (ip: 0x7fff885cae1b)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
#rqatrend#29 at /home/b/b301089/.julia/packages/RQADeforestation/e3X1u/src/analysis.jl:67
rqatrend at /home/b/b301089/.julia/packages/RQADeforestation/e3X1u/src/analysis.jl:67
unknown function (ip: 0x7ffb300fadeb)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
jl\_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/julia.h:1982 \[inlined\]
do\_apply at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/builtins.c:768
pyjlany\_call at /home/b/b301089/.julia/packages/PythonCall/flx5V/src/JlWrap/any.jl:40
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
\_pyjl\_callmethod at /home/b/b301089/.julia/packages/PythonCall/flx5V/src/JlWrap/base.jl:73
\_pyjl\_callmethod at /home/b/b301089/.julia/packages/PythonCall/flx5V/src/JlWrap/C.jl:63
unknown function (ip: 0x7fff8855cae1)
\_jl\_invoke at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:2895 \[inlined\]
ijl\_apply\_generic at /cache/build/builder-amdci4-0/julialang/julia-release-1-dot-10/src/gf.c:3077
jlcapi\_\_pyjl\_callmethod\_9063 at /home/b/b301089/.julia/compiled/v1.10/PythonCall/WdXsa\_arKfh.so (unknown line)
unknown function (ip: 0x5555556946df)
\_PyObject\_MakeTpCall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyObject\_FastCallDictTstate at python (unknown line)
\_PyObject\_Call\_Prepend at python (unknown line)
unknown function (ip: 0x55555576b7b8)
\_PyObject\_MakeTpCall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555572f6a1)
PyEval\_EvalCode at python (unknown line)
unknown function (ip: 0x555555736c2f)
unknown function (ip: 0x555555694d13)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555573930c)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555573930c)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555573930c)
unknown function (ip: 0x55555574ead9)
unknown function (ip: 0x55555569f41e)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x5555556a0580)
PyObject\_Call at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x5555556a0580)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555573930c)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555573930c)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555573930c)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555573930c)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555573930c)
task\_step\_impl at /usr/local/src/conda/python-3.10.10/Modules/\_asynciomodule.c:2653 \[inlined\]
task\_step at /usr/local/src/conda/python-3.10.10/Modules/\_asynciomodule.c:2950
unknown function (ip: 0x55555569412a)
unknown function (ip: 0x5555557bb060)
unknown function (ip: 0x55555564fb85)
unknown function (ip: 0x55555568c492)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x5555556a0580)
\_PyEval\_EvalFrameDefault at python (unknown line)
unknown function (ip: 0x55555572f6a1)
PyEval\_EvalCode at python (unknown line)
unknown function (ip: 0x555555736c2f)
unknown function (ip: 0x555555694d13)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
\_PyEval\_EvalFrameDefault at python (unknown line)
\_PyFunction\_Vectorcall at python (unknown line)
unknown function (ip: 0x55555575465a)
Py\_RunMain at python (unknown line)
Py\_BytesMain at python (unknown line)
\_\_libc\_start\_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x555555721cc0)

@ppalmes
Copy link
Author

ppalmes commented Aug 21, 2024 •
edited
Loading

temporary solution is to use PythonCall#v0.9.21

@ppalmes ppalmes changed the title error in running finalizer: UndefVarError(var=:PYNULL) V0.9.22 bug: error in running finalizer: UndefVarError(var=:PYNULL) Aug 21, 2024
@ericphanson
Copy link
Contributor

This looks like a typo:

PythonCall.jl/src/GC/GC.jl

Line 106 in 29dc91c

if any(!=(C.PYNULL), ptrs) && C.CTX.is_initialized

The tests must not insufficient since they did not catch it

@ericphanson ericphanson mentioned this issue Aug 21, 2024
Merged
@ppalmes
Copy link
Author

ppalmes commented Aug 21, 2024

good catch. very subtle bug and really spits a lot of errors. surprisingly, my package unit test doesn't catch it but overwhelms the stdout with more than 9k lines of debugging info. even the unit-testing won't flag it because the test passes. if not for the error in compiling the documentation erroring out due to the generation of large html files, it won't be caught by my unit-testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix PyNULL typo ericphanson/PythonCall.jl
3 participants
@ppalmes @ericphanson @felixcremer