Juliacall on Apple M1

[nmheim]

Hi, thank you for PythonCall.jl/juliacall! I hope it's ok to just open an issue here.
When trying to import juliacall on Apple M1 I am getting the following error:

Python 3.9.12 (main, Apr  5 2022, 01:52:34) 
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import juliacall
fatal: error thrown and no exception handler available.
ErrorException("could not load symbol "jl_n_threads":
dlsym(RTLD_DEFAULT, jl_n_threads): symbol not found")
ijl_errorf at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_dlsym at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_load_and_lookup at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
nthreads at ./threadingconstructs.jl:23 [inlined]
__preinit_threads__ at ./task.jl:706
jfptr___preinit_threads___40244 at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
_finish_julia_init at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
julia_init at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_init at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)

I would be really happy about any help with this. Thanks again! :)

[cjdoris]

You're using Julia 1.8, which isn't released yet. I'm afraid I don't have time to support dev versions of Julia. If it's still a problem with the released version, please reopen the issue.

[nmheim]

Ahh, right, I was using 1.8 because of random Zygote segfaults with 1.7. Thanks for pointing this out. It works with 1.7.

[MilesCranmer]

Is there a work-around for this? Julia 1.8 (which became a release candidate May 27), is the only available binary for M-series macs so I'm also dealing with this.

I'm evaluating whether I could port PySR to PythonCall from PyJulia but need to have it working for M-series.

[cjdoris]

JuliaCall works fine with 1.8.0-rc1 on other platforms, so the problem is specific to the M-series. Since that is already an experimental platform for Julia, I don't have the time to support it. But I'm happy to take PRs if you find ways to improve compatibility.

[gnuille]

I'm investigating this issue,
Using PDB I have been able to locate and isolate the issue although I don't understand why the crash is caused.

The line that crashes is:
https://github.com/cjdoris/PythonCall.jl/blob/73ae3059032c6951f9f0856494add524e798c05a/pysrc/juliacall/__init__.py#L124

A minimal example that reproduces the issue:

import ctypes
lib = ctypes.CDLL("/Applications/Julia-1.8.app/Contents/Resources/julia/lib/libjulia.1.8.dylib")
lib.jl_init__threading()

Any clue?

[MilesCranmer]

@gnuille thanks for posting. What happens to juliacall on M1 if you just turn that line off? Is it needed to run things?

@cjdoris Are you sure that PythonCall works on Julia 1.8-rc1 on other systems? The symbol jl_init__threading is removed from Julia 1.8 onwards, so I think that calling it should break on any system, not just macOS: https://github.com/JuliaLang/julia/blob/762561c3057b4bd541d183ca1d1b70b9945bff22/src/jlapi.c#L86-L90

[gnuille]

@MilesCranmer it segfaults :(

[dpinol]

@cjdoris Are you sure that PythonCall works on Julia 1.8-rc1 on other systems? The symbol jl_init__threading is removed from Julia 1.8 onwards, so I think that calling it should break on any system, not just macOS: https://github.com/JuliaLang/julia/blob/762561c3057b4bd541d183ca1d1b70b9945bff22/src/jlapi.c#L86-L90

yes, it works great in linux with 1.8c-1. Anyway, the same issue occurs when calling jl_init