Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

glibc cross-compilation breaks the ABI #3291

Closed
LemonBoy opened this issue Sep 22, 2019 · 2 comments
Closed

glibc cross-compilation breaks the ABI #3291

LemonBoy opened this issue Sep 22, 2019 · 2 comments
Labels
arch-aarch64 64-bit ARM bug Observed behavior contradicts documented or intended behavior
Milestone
0.5.0

Comments

@LemonBoy
Copy link
Contributor

LemonBoy commented Sep 22, 2019 •
edited
Loading

#3288 is a clear example of this. Since glibc's fstat is just a thin wrapper around __fxstat (this is done in order to let glibc change the stat struct in a backwards-compatible way) we ship a small stub in fstat.c.

Link to the glibc implementation.

For some weird reason that has everything to do with the include order passed to the cc process (or a mismatch between the version declared in the headers vs the one used by the host glibc) we end up picking a _STAT_VER from the host system instead of using the correct one as defined in the target glibc's headers.

AArch64's glibc is not happy of this and returns EINVAL.
@andrewrk andrewrk added this to the 0.5.0 milestone Sep 22, 2019
@andrewrk andrewrk added the bug Observed behavior contradicts documented or intended behavior label Sep 22, 2019
andrewrk added a commit that referenced this issue Sep 25, 2019
@andrewrk
disable known-to-be-failing tests for linux-aarch64-gnueabihf
dc3c419 
See #3288. Root cause appears to be #3291.
@andrewrk
Copy link
Member

andrewrk commented Sep 27, 2019 •
edited
Loading

goodness, how many different versioning schemes does glibc have at once?

I think a reasonable workaround for this would be to bump the default target glibc version on aarch64 to whatever was the first one where _STAT_VER equals what it does in 2.30 header files.

To fix it for older glibc versions will require patching header files to check the target glibc version and define _STAT_VER accordingly.

andrewrk added a commit that referenced this issue Sep 27, 2019
@andrewrk
std.os.fstat: EINVAL => unreachable
dc29916 
Related: #3291
@andrewrk
Copy link
Member

OK I actually tested the above thing and the problem is something else. For reference here's command line when compiling glibc fstat.c:

/home/andy/dev/zig/build/zig
cc
-MD
-MV
-MF
/home/andy/.local/share/zig/stage1/tmp/xIU5odu3aHob-fstat.o.d
-nostdinc
-fno-spell-checking
-isystem
/home/andy/dev/zig/lib/include
-target
aarch64v8-unknown-linux-gnu
-g
-fno-omit-frame-pointer
-D_DEBUG
-fno-stack-protector
-fPIC
-o
/home/andy/.local/share/zig/stage1/tmp/xIU5odu3aHob-fstat.o
-c
/home/andy/dev/zig/lib/libc/glibc/io/fstat.c
-std=gnu11
-fgnu89-inline
-g
-O2
-fmerge-all-constants
-fno-stack-protector
-fmath-errno
-ftls-model=initial-exec
-Wno-ignored-attributes
-I
/home/andy/dev/zig/lib/libc/glibc/include
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/unix/sysv/linux/aarch64
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/aarch64/nptl
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/unix/sysv/linux/include
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/unix/sysv/linux
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/nptl
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/pthread
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/unix/sysv
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/unix/aarch64
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/unix
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/aarch64
-I
/home/andy/dev/zig/lib/libc/glibc/sysdeps/generic
-I
/home/andy/dev/zig/lib/libc/glibc
-I
/home/andy/dev/zig/lib/libc/include/aarch64-linux-gnu
-I
/home/andy/dev/zig/lib/libc/include/generic-glibc
-I
/home/andy/dev/zig/lib/libc/include/aarch64-linux-any
-I
/home/andy/dev/zig/lib/libc/include/any-linux-any
-D_LIBC_REENTRANT
-include
/home/andy/dev/zig/lib/libc/glibc/include/libc-modules.h
-DMODULE_NAME=libc
-Wno-nonportable-include-path
-include
/home/andy/dev/zig/lib/libc/glibc/include/libc-symbols.h
-DPIC
-DLIBC_NONSHARED=1
-DTOP_NAMESPACE=glibc

If you look at a grep of where _STAT_VER is defined:

../lib/libc/glibc/sysdeps/unix/sysv/linux/ia64/bits/stat.h:#define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/ia64/bits/stat.h:#define _STAT_VER_LINUX		1
../lib/libc/glibc/sysdeps/unix/sysv/linux/ia64/bits/stat.h:#define _STAT_VER		_STAT_VER_LINUX
../lib/libc/glibc/sysdeps/unix/sysv/linux/mips/bits/stat.h:#define _STAT_VER_LINUX_OLD	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/mips/bits/stat.h:#define _STAT_VER_KERNEL	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/mips/bits/stat.h:#define _STAT_VER_SVR4		2
../lib/libc/glibc/sysdeps/unix/sysv/linux/mips/bits/stat.h:#define _STAT_VER_LINUX		3
../lib/libc/glibc/sysdeps/unix/sysv/linux/mips/bits/stat.h:#define _STAT_VER		_STAT_VER_LINUX /* The one defined below.  */
../lib/libc/glibc/sysdeps/unix/sysv/linux/x86/bits/stat.h:# define _STAT_VER_LINUX_OLD	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/x86/bits/stat.h:# define _STAT_VER_KERNEL	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/x86/bits/stat.h:# define _STAT_VER_SVR4		2
../lib/libc/glibc/sysdeps/unix/sysv/linux/x86/bits/stat.h:# define _STAT_VER_LINUX	3
../lib/libc/glibc/sysdeps/unix/sysv/linux/x86/bits/stat.h:# define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/x86/bits/stat.h:# define _STAT_VER_LINUX	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/x86/bits/stat.h:#define _STAT_VER		_STAT_VER_LINUX
../lib/libc/glibc/sysdeps/unix/sysv/linux/alpha/bits/stat.h:#define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/alpha/bits/stat.h:#define _STAT_VER_GLIBC2	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/alpha/bits/stat.h:#define _STAT_VER_GLIBC2_1	2
../lib/libc/glibc/sysdeps/unix/sysv/linux/alpha/bits/stat.h:#define _STAT_VER_KERNEL64	3
../lib/libc/glibc/sysdeps/unix/sysv/linux/alpha/bits/stat.h:#define _STAT_VER_GLIBC2_3_4	3
../lib/libc/glibc/sysdeps/unix/sysv/linux/alpha/bits/stat.h:#define _STAT_VER_LINUX		3
../lib/libc/glibc/sysdeps/unix/sysv/linux/alpha/bits/stat.h:#define _STAT_VER		_STAT_VER_LINUX
../lib/libc/glibc/sysdeps/unix/sysv/linux/m68k/bits/stat.h:#define _STAT_VER_LINUX_OLD	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/m68k/bits/stat.h:#define _STAT_VER_KERNEL	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/m68k/bits/stat.h:#define _STAT_VER_SVR4		2
../lib/libc/glibc/sysdeps/unix/sysv/linux/m68k/bits/stat.h:#define _STAT_VER_LINUX		3
../lib/libc/glibc/sysdeps/unix/sysv/linux/m68k/bits/stat.h:#define _STAT_VER		_STAT_VER_LINUX/* The one defined below.  */
../lib/libc/glibc/sysdeps/unix/sysv/linux/microblaze/bits/stat.h:#define _STAT_VER_LINUX_OLD    1
../lib/libc/glibc/sysdeps/unix/sysv/linux/microblaze/bits/stat.h:#define _STAT_VER_KERNEL       1
../lib/libc/glibc/sysdeps/unix/sysv/linux/microblaze/bits/stat.h:#define _STAT_VER_SVR4         2
../lib/libc/glibc/sysdeps/unix/sysv/linux/microblaze/bits/stat.h:#define _STAT_VER_LINUX        3
../lib/libc/glibc/sysdeps/unix/sysv/linux/microblaze/bits/stat.h:#define _STAT_VER              _STAT_VER_LINUX  /* The one defined below.  */
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER_LINUX	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER		_STAT_VER_LINUX
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER_LINUX_OLD	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER_KERNEL	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER_SVR4		2
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER_LINUX	3
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER		_STAT_VER_LINUX
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER_LINUX_OLD	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER_KERNEL	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER_SVR4		2
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER_LINUX		3
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER		_STAT_VER_LINUX	/* The one defined below.  */
../lib/libc/glibc/sysdeps/unix/sysv/linux/sparc/bits/stat.h:#define _STAT_VER_LINUX_OLD	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/sparc/bits/stat.h:#define _STAT_VER_KERNEL	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/sparc/bits/stat.h:#define _STAT_VER_SVR4		2
../lib/libc/glibc/sysdeps/unix/sysv/linux/sparc/bits/stat.h:#define _STAT_VER_LINUX		3
../lib/libc/glibc/sysdeps/unix/sysv/linux/sparc/bits/stat.h:#define _STAT_VER		_STAT_VER_LINUX/* The one defined below.  */
../lib/libc/glibc/sysdeps/unix/sysv/linux/powerpc/bits/stat.h:#define _STAT_VER_LINUX_OLD	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/powerpc/bits/stat.h:#define _STAT_VER_KERNEL	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/powerpc/bits/stat.h:#define _STAT_VER_SVR4		2
../lib/libc/glibc/sysdeps/unix/sysv/linux/powerpc/bits/stat.h:#define _STAT_VER_LINUX	  3
../lib/libc/glibc/sysdeps/unix/sysv/linux/powerpc/bits/stat.h:# define _STAT_VER		_STAT_VER_LINUX
../lib/libc/glibc/sysdeps/unix/sysv/linux/powerpc/bits/stat.h:# define _STAT_VER		_STAT_VER_KERNEL
../lib/libc/glibc/sysdeps/unix/sysv/linux/generic/bits/stat.h:#define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/generic/bits/stat.h:#define _STAT_VER_LINUX		0
../lib/libc/glibc/sysdeps/unix/sysv/linux/generic/bits/stat.h:#define _STAT_VER		_STAT_VER_KERNEL

You can see that the directory getting picked up is:

../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER_LINUX_OLD	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER_KERNEL	1
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER_SVR4		2
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER_LINUX		3
../lib/libc/glibc/sysdeps/unix/sysv/linux/bits/stat.h:#define _STAT_VER		_STAT_VER_LINUX	/* The one defined below.  */

For a value of 3. But the correct value is 0:

andy@ark ~> grep -RI '#define.*_STAT_VER' ~/Downloads/glibc/multi/install/glibcs/aarch64-linux-gnu/usr/include/
/home/andy/Downloads/glibc/multi/install/glibcs/aarch64-linux-gnu/usr/include/bits/stat.h:#define _STAT_VER_KERNEL	0
/home/andy/Downloads/glibc/multi/install/glibcs/aarch64-linux-gnu/usr/include/bits/stat.h:#define _STAT_VER_LINUX		0
/home/andy/Downloads/glibc/multi/install/glibcs/aarch64-linux-gnu/usr/include/bits/stat.h:#define _STAT_VER		_STAT_VER_KERNEL

I'm confused because the only ways it gets defined to 0 are these:

[nix-shell:~/dev/zig/build]$ grep -RI '_STAT_VER.*\b0\b' ../lib/libc/glibc/
../lib/libc/glibc/io/sys/stat.h:# define _STAT_VER	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/ia64/bits/stat.h:#define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/x86/bits/stat.h:# define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/alpha/bits/stat.h:#define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/s390/bits/stat.h:# define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/generic/bits/stat.h:#define _STAT_VER_KERNEL	0
../lib/libc/glibc/sysdeps/unix/sysv/linux/generic/bits/stat.h:#define _STAT_VER_LINUX		0

Is sysdeps/unix/sysv/linux/generic supposed to go before sysdeps/unix/sysv/linux? I'll have to check a build of glibc to find out. Found one:

aarch64-unknown-linux-gnu-gcc

fstat.c
-c
-std=gnu99
-fgnu89-inline

-O2
-Wall
-Winline
-Wwrite-strings
-fmerge-all-constants
-fno-strict-aliasing
-frounding-math
-pipe
-Wstrict-prototypes
-fPIC
-I../include
-I/var/tmp/portage/sys-libs/glibc-2.17/work/build-default-aarch64-unknown-linux-gnu-nptl/io
-I/var/tmp/portage/sys-libs/glibc-2.17/work/build-default-aarch64-unknown-linux-gnu-nptl
-I../ports/sysdeps/unix/sysv/linux/aarch64/nptl
-I../ports/sysdeps/unix/sysv/linux/aarch64
-I../ports/sysdeps/unix/sysv/linux/generic
-I../sysdeps/unix/sysv/linux/wordsize-64
-I../nptl/sysdeps/unix/sysv/linux
-I../nptl/sysdeps/pthread
-I../sysdeps/pthread
-I../ports/sysdeps/unix/sysv/linux
-I../sysdeps/unix/sysv/linux
-I../sysdeps/gnu
-I../sysdeps/unix/inet
-I../nptl/sysdeps/unix/sysv
-I../ports/sysdeps/unix/sysv
-I../sysdeps/unix/sysv
-I../nptl/sysdeps/unix
-I../ports/sysdeps/unix
-I../sysdeps/unix
-I../sysdeps/posix
-I../ports/sysdeps/aarch64/fpu
-I../ports/sysdeps/aarch64/nptl
-I../ports/sysdeps/aarch64
-I../sysdeps/wordsize-64
-I../sysdeps/ieee754/ldbl-128
-I../sysdeps/ieee754/dbl-64/wordsize-64
-I../sysdeps/ieee754/dbl-64
-I../sysdeps/ieee754/flt-32
-I../ports/sysdeps/aarch64/soft-fp
-I../sysdeps/ieee754
-I../sysdeps/generic
-I../nptl
-I../ports

-I..
-I../libio
-I.
-nostdinc
-isystem
/usr/lib/gcc/aarch64-unknown-linux-gnu/4.9.0-pre9999/include
-isystem
/usr/lib/gcc/aarch64-unknown-linux-gnu/4.9.0-pre9999/include-fixed
-isystem
/usr/include
-D_LIBC_REENTRANT
-include
../include/libc-symbols.h

-DPIC
-DLIBC_NONSHARED=1

-o
/var/tmp/portage/sys-libs/glibc-2.17/work/build-default-aarch64-unknown-linux-gnu-nptl/io/fstat.oS
-MD
-MP
-MF
/var/tmp/portage/sys-libs/glibc-2.17/work/build-default-aarch64-unknown-linux-gnu-nptl/io/fstat.oS.dt
-MT
/var/tmp/portage/sys-libs/glibc-2.17/work/build-default-aarch64-unknown-linux-gnu-nptl/io/fstat.oS

Apparently it is. OK.

@andrewrk andrewrk added the arch-aarch64 64-bit ARM label Sep 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-aarch64 64-bit ARM bug Observed behavior contradicts documented or intended behavior
Projects
None yet
Milestone
0.5.0
Development

No branches or pull requests
2 participants
@LemonBoy @andrewrk