Julia nightly failure to precompile REPL

[fingolfin]

In the most recent run of the GAP.jl CI runs with Julia nightly it fails to precompile REPL. Note that GAP.jl has REPL in its dependencies.

For example this log:

  12985.1 ms  ✓ AbstractAlgebra
           ✗ REPL
  28704.7 ms  ✓ Parsers
   1370.0 ms  ✓ JSON
           ✗ Pkg → REPLExt
    462.3 ms  ✓ GAP_lib_jll
    618.2 ms  ✓ GAP_pkg_juliainterface_jll
   3727.9 ms  ✓ Aqua
           ✗ Documenter
           ✗ GAP
  21 dependencies successfully precompiled in 65 seconds. 32 already precompiled.

ERROR: LoadError: The following 3 direct dependencies failed to precompile:

GAP --code-coverage=@/home/runner/work/GAP.jl/GAP.jl --color=yes --check-bounds=yes --warn-overwrite=yes --depwarn=error --inline=yes --startup-file=no --track-allocation=none --check-bounds=yes --compiled-modules=yes 

Failed to precompile GAP [c863536a-3901-11e9-33e7-d5cd0df7b904] to "/home/runner/.julia/compiled/v1.12/GAP/jl_ST05dz".
WARNING: Method definition formatdoc(Base.Docs.DocStr) in module REPL at /opt/hostedtoolcache/julia/nightly/x64/share/julia/stdlib/v1.12/REPL/src/docview.jl:79 overwritten in module REPL at /cache/build/builder-amdci4-3/julialang/julia-master/usr/share/julia/stdlib/v1.12/REPL/src/docview.jl:79.
ERROR: Method overwriting is not permitted during Module precompilation. Use `__precompile__(false)` to opt-out of precompilation.
┌ Error: Unexpected error (Review REPL precompilation with debug_output on):
│ 2004lERROR: Error when precompiling module, potentially caused by a __precompile__(false) declaration in the module.
└ @ REPL.Precompile /opt/hostedtoolcache/julia/nightly/x64/share/julia/stdlib/v1.12/REPL/src/precompile.jl:26
ERROR: LoadError: Failed to precompile REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb] to "/home/runner/.julia/compiled/v1.12/REPL/jl_QGthvI".
Stacktrace:
...

[fingolfin]

@IanButterworth might this be related to your PR #55782?

[IanButterworth]

Why is it recompiling REPL and not using the bundled one? Are you overriding JULIA_DEPOT_PATH without the trailing : ?

[fingolfin]

No, we are not messing with depot paths. I have no idea why it might recompile REPL

[benlorenz]

I can reproduce this locally as well (with Polymake), the package loads fine when doing using Polymake (without precompiling REPL) but running ]test triggers the failing precompilation of REPL.

Setting JULIA_DEBUG=loading shows:

┌ Debug: Rejecting cache file /home/lorenz/software/polymake/julia/julia/julia-nightly/share/julia/compiled/v1.12/REPL/u0gqU_eQk43.ji because uuid mapping for Base.PkgId(nothing, "") => Base.PkgId(Base.UUID("3fa0cd96-eef1-5676-8a61-b3b8758bbffb"), "REPL") has changed, expected Base.PkgId(nothing, "") => nothing
└ @ Base loading.jl:3859
┌ Debug: Rejecting cache file /home/lorenz/software/polymake/julia/julia/julia-nightly/share/julia/compiled/v1.12/REPL/u0gqU_OrKMu.ji for Base.PkgId(nothing, "") since the flags are mismatched
│   requested flags: use_pkgimages = true, debug_level = 1, check_bounds = 1, inline = true, opt_level = 2 [171]
│   cache file:      use_pkgimages = true, debug_level = 1, check_bounds = 0, inline = true, opt_level = 2 [163]
└ @ Base loading.jl:3740
Precompiling for configuration --code-coverage=none --color=yes --check-bounds=yes --warn-overwrite=yes --depwarn=yes --inline=yes --startup-file=no --track-allocation=none
...

I added the debug flag to our CI here:
https://github.com/oscar-system/Polymake.jl/actions/runs/10924713958/job/30324438870?pr=497#step:8:601

Edit: Small example package (https://github.com/benlorenz/MyReplTest.jl) with only REPL as a dependency. Failing CI job: https://github.com/benlorenz/MyReplTest.jl/actions/runs/10925420997/job/30326858029#step:4:173

[vtjnash]

It looks like it believes it was requested to load 2 different copies of REPL, but apparently REPL has a type-piracy bug, so that is not permitted:

/opt/hostedtoolcache/julia/nightly/x64/share/julia/stdlib/v1.12/REPL/src/docview.jl:79 overwritten in module REPL at /cache/build/builder-amdci4-3/julialang/julia-master/usr/share/julia/stdlib/v1.12/REPL/src/docview.jl:79

[vtjnash]

AFAICT, the error occurs because the precompile script starts with trying to execute

import REPL
REPL.activate(REPL.Precompile; interactive_utils=false)

And whether or not that succeeds depends on whether those global mutations are able to succeed before the activate call works. We should probably move that into the atreplinit callback, so it runs inside REPL.Precompile before the Base.run_std_repl call instead of in Main afterwards

[IanButterworth]

That doesn't work out of the box because mistate === nothing at the time atreplinit is evaluated

julia/stdlib/REPL/src/REPL.jl

Line 763 in 02549d5

mistate === nothing && return nothing

[IanButterworth]

@fingolfin @benlorenz can you confirm that this is fixed on your CI. Both the error and the fact that REPL was being precompiled at all (Pkg.test should be using the bundled stdlib caches)

[benlorenz]

Yes, CI for Polymake and GAP is working fine again, thanks for the quick fix!

I don't see any messages about precompiling REPL, and the debug log for my test-package shows the bundled file being loaded:

     Testing Running tests...
...
┌ Debug: Loading object cache file /opt/hostedtoolcache/julia/nightly/x64/share/julia/compiled/v1.12/REPL/u0gqU_HKgpq.so for REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb]
└ @ Base loading.jl:1240
┌ Debug: Loading object cache file /home/runner/.julia/compiled/v1.12/MyReplTest/hmoKa_UaWUl.so for MyReplTest [269616a3-f737-4ecc-9db8-f4988c691a26]
└ @ Base loading.jl:1240
Test Summary: | Pass  Total  Time
a             |    1      1  0.0s
     Testing MyReplTest tests passed 

https://github.com/benlorenz/MyReplTest.jl/actions/runs/10925420997/job/30377816478#step:4:200

[IanButterworth]

Great. Thanks