Bundled GCC on windows doesn't pick up system link paths

[alexcrichton]

A system mingw-w64 gcc installation has many many libs in /mingw64/$triple/lib which gcc will make available for linking by default. The bundled GCC that rust provides, however, does not link against these libraries by default (they are not in any -L flag). This means that gcc invocations which would otherwise work on the system begin to fail because the bundled GCC is preferred.

For a concrete example, see rust-lang/cargo#620.

cc @vadimcn

[alexcrichton]

This has also broken the cargo nightlies and builds

error: linking with `gcc` failed: exit code: 1
note: gcc '-m64' '-L' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib' '-o' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\cargo.exe' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\cargo.o' '-Wl,--whole-archive' '-lmorestack' '-Wl,--no-whole-archive' '-fno-lto' '-fno-use-linker-plugin' '-Wl,--gc-sections' '-shared-libgcc' '-Wl,--enable-long-section-names' '-Wl,--nxcompat' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libnative-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\libcargo-3045fb9d724e9092.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libglob-54e4fdcda4a7c968.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libflate2-7571a86064e76c6d.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libterm-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libtar-6ccba66e5440ac37.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libtoml-a3516e82ea71aded.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libgit2-09174fe7d724f48b.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libtime-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libsemver-588b3e8c2b1d977b.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libdocopt-c165ee05d1fe3dd7.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\liblibgit2-fcba23a36b59f31c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libopenssl-static-sys-6ae299ea46a69e57.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libdebug-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libcurl-f3f9ef32955b72e6.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\liburl-921578b148f50e06.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libserialize-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\liblog-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libregex-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps\libencoding-4805bc5305f7cd87.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libstd-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libsync-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\librand-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\librustrt-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libcollections-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\liballoc-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\liblibc-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libunicode-4e7c5e5c.rlib' 'C:\bot\slave\cargo-win64-64\build\rustc\bin\rustlib\x86_64-w64-mingw32\lib\libcore-4e7c5e5c.rlib' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\deps' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\flate2-7571a86064e76c6d' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\curl-sys-9d9d6a6df3c98f14' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\libgit2-fcba23a36b59f31c' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\openssl-static-sys-6ae299ea46a69e57' '-L' 'C:\bot\slave\cargo-win64-64\build\target\x86_64-w64-mingw32\native\libssh2-static-sys-c29829f2cf163ffd' '-L' 'C:\bot\slave\cargo-win64-64\build\.rust' '-L' 'C:\bot\slave\cargo-win64-64\build' '-Wl,--whole-archive' '-Wl,-Bstatic' '-Wl,--no-whole-archive' '-Wl,-Bdynamic' '-lws2_32' '-lkernel32' '-lwinhttp' '-lrpcrt4' '-lole32' '-lws2_32' '-lbcrypt' '-lcrypt32' '-lws2_32' '-lgcc_s' '-lcompiler-rt'
note: ld: cannot find -lwinhttp
ld: cannot find -lrpcrt4
ld: cannot find -lole32
ld: cannot find -lbcrypt

[brson]

I'm not sure this is a bug if cargo is relying on random mingw libraries - eventually we will switch away from mingw completely and those libs will not be available.

[vadimcn]

How did cargo machine end up with a bundled gcc? Does it use windows installer?!

[alexcrichton]

The cargo builders do not have a pre-installed rustc and they are run in a mingw-w64 shell (and mingw32 for now). The first step of a cargo build on the bots is to download the latest rust nightly installer and unpack it.

[vadimcn]

Hmm, I'd have expected them to use a snapshot compiler...

[jhasse]

@brson What are you planning to use instead of MinGW?

[retep998]

If we want to link Rust programs against Windows libraries we'll have to either bundle those libraries, depend on the system MinGW's copy of them, or depend on the MSVC toolchain (not likely to happen). So unless Cargo dependencies are going to start bundling the system libraries, I don't see how we can really avoid a dependency on a system MinGW for any non-trivial project.

[vadimcn]

Well, we could:

1. Tell users that if a program uses anything other than standard Rust libraries, they need to install a full mingw toolchain. This is the current status-quo.
2. Bundle more (all?) of mingw-provided import libraries. This bloats the download package though, which is why I didn't do this.
3. Tell users to use direct linking to dlls (see here). Unfortunately, this option is not very ergonomic at the moment: adding %SystemRoot%\System32 to the library search path does not work because ld chokes on kernel32.dll. However if the relevant system dlls are copied out to a separate folder, direct linking seems to work fine.
   3a. Figure out how to make gcc find bundled import libraries before searching System32.
4. Tell users to use dlltool utility (which we bundle) to create import libraries themselves.
   4a. Teach rustc or cargo to do it for them.

[brson]

Nominating. From what I understand Cargo's mingw dependencies are preventing us from accomplishing #11782.

[brson]

@jhasse We rely on mingw principally for the GNU linker, and would prefer to instead use the msvc linker.

[brson]

I'm not clear on whether the problem is cargo linking to windows libs or gnu libs. Shouldn't we be trying to link to windows libs since those don't depend on mingw?

[brson]

@vadimcn Not fully understanding the problem here, my strong preference is for your option 1: we should not be encouraging mingw dependencies; much of the work we are doing here is explicitly to get away from the mingw dependency.

[brson]

From what @alexcrichton says, cargo should work without mingw, which is what we need: the problem it has is that it can't build without mingw, which is just a problem for our bots. We should test this.

[vadimcn]

@brson: Not sure what you mean by "windows" libs.

[vadimcn]

We rely on mingw principally for the GNU linker, and would prefer to instead use the msvc linker.

@brson: I doubt that. Windows does not ship with linker. It's only available if you install MSVC toolchain. I doubt we'd be allowed to extract and redistribute it the way we do it with gcc.

[jhasse]

I'm also strongly against MSVC if that would imply a dependency on the VC++ runtime (llvm is using C++, right?).

[retep998]

Using MinGW we already have a dependency on the VC++ runtime, just an ancient version that all Windows computers already have installed. If we switched to the msvc toolchain we'd likely use a newer VC++ runtime, so if we linked to it dynamically and didn't include the dlls then some people might have to install the VC++ redistributable.

[pnkfelix]

@alexcrichton says this is actually fixed or at least working "fine". Closing.