julia build error on redhat

[mahi83]

I tried everything possible but I can't figure out why I am getting this error:

[nalimilan]

What RedHat distribution exactly? Also, please post a longer log excerpt somewhere (e.g. as a GitHub gist).

[mahi83]

These are the following versions I own:
Centos: 6.3
openssl: 1.0.2i
cmake: 3.7.0
gcc: 6.3.0
python: 2.7.0

[  1%] Linking C executable libgit2_clar
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_globallookup':
dso_dlfcn.c:(.text+0x11): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x24): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x2f): undefined reference to `dlclose'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_bind_func':
dso_dlfcn.c:(.text+0x334): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x3db): undefined reference to `dlerror'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_bind_var':
dso_dlfcn.c:(.text+0x45b): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x514): undefined reference to `dlerror'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_load':
dso_dlfcn.c:(.text+0x578): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x5db): undefined reference to `dlclose'
dso_dlfcn.c:(.text+0x613): undefined reference to `dlerror'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_pathbyaddr':
dso_dlfcn.c:(.text+0x69f): undefined reference to `dladdr'
dso_dlfcn.c:(.text+0x701): undefined reference to `dlerror'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_unload':
dso_dlfcn.c:(.text+0x752): undefined reference to `dlclose'
collect2: error: ld returned 1 exit status
make[4]: *** [libgit2_clar] Error 1
make[3]: *** [CMakeFiles/libgit2_clar.dir/all] Error 2
make[2]: *** [all] Error 2
make[1]: *** [build/libgit2-211e117a0590583a720c53172406f34186c543bd/libgit2.so] Error 2
make: *** [julia-deps] Error 2

[ararslan]

Looks like -ldl isn't getting passed somewhere.

[mahi83]

What should I do to check if it isnt being passed?

[ararslan]

I'm not entirely sure. It isn't something that the user should have to be doing; it should be in a makefile somewhere.

Are you building Julia from the master branch? If you do make -C deps distclean-libgit2 then try again, do you get the same error?

[mahi83]

Does this mean it worked?

make: Entering directory `/mnt/store2/home/karim1/tmp/julia-0.5.0/deps'
rm -rf /mnt/store2/home/karim1/tmp/julia-0.5.0/deps/srccache/libgit2-211e117a059                               0583a720c53172406f34186c543bd /mnt/store2/home/karim1/tmp/julia-0.5.0/deps/srcca                               che/libgit2-211e117a0590583a720c53172406f34186c543bd.tar.gz build/libgit2 build/                               libgit2-211e117a0590583a720c53172406f34186c543bd
make: Leaving directory `/mnt/store2/home/karim1/tmp/julia-0.5.0/deps'

[mahi83]

It didn't work. This is the error I get when I run make install

[ 79%] Linking C executable libgit2_clar
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_globallookup':
dso_dlfcn.c:(.text+0x11): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x24): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x2f): undefined reference to `dlclose'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_bind_func':
dso_dlfcn.c:(.text+0x334): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x3db): undefined reference to `dlerror'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_bind_var':
dso_dlfcn.c:(.text+0x45b): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x514): undefined reference to `dlerror'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_load':
dso_dlfcn.c:(.text+0x578): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x5db): undefined reference to `dlclose'
dso_dlfcn.c:(.text+0x613): undefined reference to `dlerror'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_pathbyaddr':
dso_dlfcn.c:(.text+0x69f): undefined reference to `dladdr'
dso_dlfcn.c:(.text+0x701): undefined reference to `dlerror'
/export/home/karim1/openssl-1.0.2i/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_unload':
dso_dlfcn.c:(.text+0x752): undefined reference to `dlclose'
collect2: error: ld returned 1 exit status
make[5]: *** [libgit2_clar] Error 1
make[4]: *** [CMakeFiles/libgit2_clar.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [build/libgit2-211e117a0590583a720c53172406f34186c543bd/libgit2.so] Error 2
make[1]: *** [julia-deps] Error 2
make: *** [install] Error 2

[ararslan]

1. What's the output of echo $LD_LIBRARY_PATH?
2. What are the contents of your Make.user file, if anything?
3. Why is there an OpenSSL build directory in your home directory?

[mahi83]

1. /export/home/karim1/libgit2-0.25.0/lib:/export/home/karim1/openssl-1.0.2i/lib:/export/home/karim1/Python-2.7.9/lib:/export/home/karim1/gcc-6.3.0/lib64:/export/home/karim1/gcc-6.3.0/lib:/opt/openmpi/lib:/opt/python/lib

2. OPENBLAS_DYNAMIC_ARCH=0

3. It is in my home directory because thats where I installed OpenSSL.

[mahi83]

anybody?

[mahi83]

Is there a way to specify the location of libgit2 when running make?

[ihnorton]

Is there a way to specify the location of libgit2 when running make?

I don't think there are any compile-time dependencies on libgit, so setting USE_SYSTEM_LIBGIT2=1 in Make.user should avoid building it. Then Julia will look for libgit2.so at runtime (assuming it is in a standard place).

(related: #17920)

[ararslan]

In particular, the libgit2 in your home directory is at the front of your LD_LIBRARY_PATH, so if you use USE_SYSTEM_LIBGIT2 then it will likely pick up the right one (assuming that's the one you want).

[mahi83]

I tried doing USE_SYSTEM_LIBGIT2 and Julia seems to build ok. However, now when I try installing a package like DataFrames I get the following error:

julia> Pkg.add("DataFrames")
INFO: Initializing package repository /export/home/karim1/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ECERTIFICATE, Class:SSL, The SSL certificate is invalid)
 in macro expansion at ./libgit2/error.jl:99 [inlined]
 in clone(::String, ::String, ::Base.LibGit2.CloneOptions) at ./libgit2/repository.jl:191
 in #clone#109(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredentials}, ::Function, ::String, ::String) at ./libgit2/libgit2.jl:327
 in (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::String) at ./<missing>:0
 in (::Base.Pkg.Dir.##4#6{String,String})() at ./pkg/dir.jl:49
 in cd(::Base.Pkg.Dir.##4#6{String,String}, ::String) at ./file.jl:59
 in init(::String, ::String) at ./pkg/dir.jl:47
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{Any,N}) at ./pkg/dir.jl:28
 in add(::String) at ./pkg/pkg.jl:100

I believe its not detecting the right path for libgit2. Any suggestions?

[ihnorton]

You need a proper root cert, and your libgit2 build needs to know where to find it. Look at the first page or two of deps/libgit2.mk. Why are you building libgit2 separately in the first place?

[mahi83]

Its not building properly if I don't do it separately. I get the error I mentioned in my first post in this thread. If you have any recommendations of fixing that, I would love to hear it. I would rather fix that then figure out how to build libgit2 separately.

[ihnorton]

Try adding -ldl to CMAKE_C_FLAGS in deps/libgit2.mk. Is it correct that you are building the release-0.5 branch against your own OpenSSL because the system version is too old?

Going forward, mbedtls is the preferred encryption SSL library on Linux, see #18658. But I think that came in just after the 0.5 release and has not been back-ported. You could try cherry-picking the commit.

In the future please ask build questions on Discourse.

[tkelman]

We don't have a minimum version requirement on openssl, libgit2 should be able to build against any old system version.

[mahi83]

Yes I am building the 0.5 release against my own OPENSSL because the system version is too old. I know you guys don't have a minimum version requirement, but I was having issues building CMAKE with the old version. Also, I was able to install Julia by adding the -ldl flag in the libgit2.mk file. I still get the package error I mentioned earlier. There is a high likelihood the issue might be with the SSL certificate. I am ok for now just using Julia on my system. I just need to run my code on several hundred cores, and I do not think it will be an issue. I can always manually install the packages (if I need them). Thank you guys for helping me.

[tkelman]

ah good point cmake may have a minimum version requirement on openssl that's stricter than libgit2. I usually use cmake's binaries if I'm on a system where the distro copy is too old, rather than building cmake from source.