Caching binaries

[SimonDanisch]

Hi,
my task is: make the first run of X as fast as the second run.
It seems quite difficult to achieve.
The situation is, that I'm covering all code I want to execute (over runtests.jl), so in theory it should be straight forward to infer what to compile.

What I've tried:

1. Including runtests.jl into user image
2. SnoopCompile + user image

use SnoopCompile to generate snoop_precompiles.jl from runtests.jl
---- MyPackage.jl
module MyPackage
include("snoop_precompiles.jl")
end
---- userimg.jl
using MyPacke
----

3. BuildExecutable

Results:

1. I have had good results with this approach, but it doesn't work for arbitrary code. Would it be worth while to trim the included code to work? I had various failures, and as soon as OpenGL comes into the game, things were not working at all anymore
2. This sometimes generates a compilation speed up, sometimes not... Sometimes it changes type inference resulting in weird errors. Is this SnoopCompiles fault, or to be expected? Would fixing SnoopCompile be well invested time?
3. The compiled executable actually executes slower than julia executable_script.jl, suggesting that this just links the exe to the JIT, while not caching any binary. I guess I still need precompile statements and it doesn't recursively infer from main() what to compile?

My dream workflow is something like this:

# this could be executed by Pkg.build() ->
>julia --cache-binary --output-incremental=yes GLVisualize/test/runtests.jl --> v0.5/lib/cache_x.so
julia> using GLVisualize; <-- all functions already run in runtests will need no JIT
>generate_py_wrapper v0.5/lib/cache_x.so GLVisualize/src/GLVisualize.jl
--> generates py c interop thingies to call the lib... Hehehe

Btw, would generating a seperate .so + ccall solve the (quite annoying) problem, that we always need to interact with the sysimg?

I'd be quite willing to work on this in some way.
I just need to know where to start :)

[tkelman]

largely a duplicate of #15864

[SimonDanisch]

I saw that issue, but it isn't really set up to answer the questions that I have right now ;)