llvm-objcopy produces 160MB binary. It used to be 74KB

[dc740]

I'm developing a platform for TockOS. They upgraded the nightly version from 03-06 to 06-03, and now llvm-objcopy produces a 160MB binary instead of a 74KB binary. I also tried on the latest nightly (06-09), and the result is the same.

I tried this code with the same .elf file:

~/.rustup/toolchains/nightly-2020-06-03-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-objcopy --output-target=binary ./edu-ciaa.elf ./edu-ciaa.06-03.bin

~/.rustup/toolchains/nightly-2020-03-06-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/llvm-objcopy --output-target=binary ./edu-ciaa.elf ./edu-ciaa.03-06.bin

I expected to see both binaries have roughly the same size, since both are applied to the same elf file. (which can be found here: https://github.com/dc740/tock/blob/fd6517da9f22d088a30d6b7cb767e42ab820814f/boards/ciaa/edu-ciaa/sample-edu-ciaa.elf )

Instead, the produced binary from nightly 03-06 is 74KB and the one produced by the newer nightly versions 06-03 and 06-19 is 160MB.

Meta

rustc --version --verbose:

rustc 1.46.0-nightly (feb3536eb 2020-06-09)
binary: rustc
commit-hash: feb3536eba10c2e4585d066629598f03d5ddc7c6
commit-date: 2020-06-09
host: x86_64-unknown-linux-gnu
release: 1.46.0-nightly
LLVM version: 10.0

[wesleywiser]

@rustbot ping llvm

Does anyone know what's going on here?

[rustbot]

Hey LLVM ICE-breakers! This bug has been identified as a good
"LLVM ICE-breaking candidate". In case it's useful, here are some
instructions for tackling these sorts of bugs. Maybe take a look?
Thanks! <3

cc @camelid @comex @cuviper @DutchGhost @dyncat @hanna-kruppe @hdhoang @heyrutvik @JOE1994 @jryans @mmilenko @nagisa @nikic @Noah-Kennedy @SiavoshZarrasvand @spastorino @vertexclique

[nagisa]

It appears that the LLVM version shipped with rustc is used. There has been a bump in LLVM version in the time range, which is most suspect here.

[spastorino]

@rustbot ping cleanup

It would be nice to find where this has regressed, as @nagisa noted, likely to be in an LLVM bump.

[rustbot]

Hey Cleanup Crew ICE-breakers! This bug has been identified as a good
"Cleanup ICE-breaking candidate". In case it's useful, here are some
instructions for tackling these sorts of bugs. Maybe take a look?
Thanks! <3

cc @AminArria @camelid @chrissimpkins @contrun @DutchGhost @elshize @ethanboxx @h-michael @HallerPatrick @hdhoang @hellow554 @imtsuki @kanru @KarlK90 @LeSeulArtichaut @MAdrianMattocks @matheus-consoli @mental32 @nmccarty @Noah-Kennedy @pard68 @PeytonT @pierreN @Redblueflame @RobbieClarken @RobertoSnap @robjtede @SarthakSingh31 @senden9 @shekohex @sinato @spastorino @turboladen @woshilapin @yerke

[spastorino]

Assigning P-high as discussed as part of the Prioritization Working Group process and removing I-prioritize.

[LeSeulArtichaut]

Regression happened between nightly-2020-05-17 and nightly-2020-05-23, continuing bisection.

[cuviper]

I can reproduce this with vanilla LLVM tools on Fedora 32.

• The input file is 2.0MB (2069404).
• With llvm-objcopy from llvm-10.0.0-1.fc32.x86_64, the output is 161M (167849124).
• With llvm-objcopy-9.0 from llvm9.0-9.0.1-8.fc32.x86_64, the output is 76K (76964).
• With llvm-objcopy-8.0 from llvm8.0-8.0.0-11.fc32.x86_64, the output is 76K (76964).

Here are the section and program headers:

$ eu-readelf -lS sample-edu-ciaa.elf
There are 23 section headers, starting at offset 0x1f9004:

Section Headers:
[Nr] Name                 Type         Addr     Off    Size   ES Flags Lk Inf Al
[ 0]                      NULL         00000000 000000 000000  0        0   0  0
[ 1] .stack               NOBITS       10000000 000114 001000  0 WA     0   0  1
[ 2] .text                PROGBITS     1a000000 000200 0112f8  0 AXM    0   0 16
[ 3] .ARM.exidx           ARM_EXIDX    1a0112f8 0114f8 000010  0 AL     2   0  4
[ 4] .storage             PROGBITS     1a011308 011508 0000f8  0 A      0   0  1
[ 5] .apps                PROGBITS     1a040000 011600 000000  0 A      0   0  1
[ 6] .relocate            PROGBITS     10001000 011600 0018a4  0 WA     0   0  4
[ 7] .sram                NOBITS       100028a4 012ea4 00575c  0 WA     0   0  4
[ 8] .debug_str           PROGBITS     00000000 012ea4 067400  1 MS     0   0  1
[ 9] .debug_loc           PROGBITS     00000000 07a2a4 043b4e  0        0   0  1
[10] .debug_abbrev        PROGBITS     00000000 0bddf2 001125  0        0   0  1
[11] .debug_info          PROGBITS     00000000 0bef17 09c23b  0        0   0  1
[12] .debug_aranges       PROGBITS     00000000 15b158 0017e8  0        0   0  8
[13] .debug_ranges        PROGBITS     00000000 15c940 00f678  0        0   0  1
[14] .debug_pubnames      PROGBITS     00000000 16bfb8 026ec5  0        0   0  1
[15] .debug_pubtypes      PROGBITS     00000000 192e7d 037bca  0        0   0  1
[16] .ARM.attributes      ARM_ATTRIBUTES 00000000 1caa47 000030  0        0   0  1
[17] .debug_frame         PROGBITS     00000000 1caa78 003018  0        0   0  4
[18] .debug_line          PROGBITS     00000000 1cda90 01da06  0        0   0  1
[19] .comment             PROGBITS     00000000 1eb496 000093  1 MS     0   0  1
[20] .symtab              SYMTAB       00000000 1eb52c 003ec0 16       22 972  4
[21] .shstrtab            STRTAB       00000000 1ef3ec 0000f1  0        0   0  1
[22] .strtab              STRTAB       00000000 1ef4dd 009b27  0        0   0  1

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz  MemSiz   Flg Align
  LOAD           0x000200 0x1a000000 0x1a000000 0x011400 0x011400 R E 0x200
  LOAD           0x011600 0x10001000 0x1a011400 0x0018a4 0x0018a4 RW  0x200
  GNU_STACK      0x000000 0x00000000 0x00000000 0x000000 0x000000 RW  0x0
  ARM_EXIDX      0x0114f8 0x1a0112f8 0x1a0112f8 0x000010 0x000010 R   0x4

 Section to Segment mapping:
  Segment Sections...
   00      [RO: .text .ARM.exidx .storage]
   01      .relocate
   02
   03      [RO: .ARM.exidx]

I notice that adding the sizes of the two load sections, 0x011400 + 0x0018a4 is 76964, precisely matching the former llvm-objcopy output size. I haven't figured out the exact match for the 161M (167849124), but it's pretty close to the virtual address range of those load segments, 0x10001000..0x1a011400 is 167838720 bytes.

[LeSeulArtichaut]

@cuviper So that would imply that the bug is coming from LLVM-upstream?

[LeSeulArtichaut]

From my testing, the regression happened in nightly-2020-05-22. @nagisa does that correspond to a LLVM bump?

[cuviper]

Regression happened between nightly-2020-05-17 and nightly-2020-05-23, continuing bisection.

The upgrade to LLVM 10 merged on 2020-05-20: #67759 (comment)
(GitHub shows that as 7:55PM PDT, which would have missed the nightly on 05-21 UTC.)

[cuviper]

I haven't tested this directly, but this change looks relevant:
llvm/llvm-project@d28c6d5
https://reviews.llvm.org/D71035

So that would imply that the bug is coming from LLVM-upstream?

Yes, although I hesitate to call it a bug. From the manpage:

If binary is used as the value for --output-target, the output file will be a raw binary file, containing the memory image of the input file. Symbols and relocation information will be discarded. The image will start at the address of the first loadable section in the output.

Since the virtual addresses really do have a large span, 161M does seem like an appropriate memory image. The old 76K size had those loaded segments adjacent, which doesn't match memory.

[spastorino]

I think we should close this issue as intended LLVM behavior given @cuviper comments. At least removing P-high label for now. @dc740 thoughts?.

[dc740]

To be honest, right now I don't have the knowledge to either decline or accept this change in behavior as a bug. I will be doing some research so I can understand what's going on, and what would be the right way to generate the .elf file with the correct virtual addresses, which, as I understand, is the culprit of the problem. Is that the right approach? Did I understand that correctly? I'd really appreciate if anyone wants to jump in and explain where to start looking, since this is currently out of my comfort zone, but feel free to close the bug report. I don't want to cause any more inconvenience.

Thanks everyone for the time and responses.

[spastorino]

Ok, I'm going to close this issue, maybe @cuviper can explain it a bit better if it's not a problem for them.

[cuviper]

I don't have experience using binary objcopy -- I just inferred what I could from the information provided. I suppose it would work if you can affect the virtual addresses of your input file. Another possibility is to use objcopy --only-section=... with just the sections you actually need for your target. I suspect you don't actually need everything, if you were working well enough without the virtual memory addressing before. For instance, --only-section=.text gives me a 69K file.