binutils/ld.bfd, -fdebug-prefix-map, FILE entries (was: Re: dietlibc; build path issue on ARM despite -fdebug-prefix-map)

[Christian Seiler]

Hi there,

On 11/05/2016 12:09 PM, Ximin Luo wrote:
> Christian Seiler:
>> It appears as though gcc hard-codes the path for the startup file
>> on armhf, while it doesn't do so on x86. A simple way to try this
>> out, even without dietlibc:
>>
>> cat > hello.c <<EOF
>> #include <stdio.h>
>>
>> int main()
>> {
>>     puts("Hello World!");
>>     return 0;
>> }
>> EOF
>> gcc -g -Wall -o hello hello.c
>> readelf -Ws hello | grep FILE
>>
>> On x86 the internal startup files (start.o for dietlibc, Scrt1.o,
>> crti.o, crtn.o, elf-init.oS for glibc) are not referenced at all,
>> while on ARM they are not only referenced, but are referenced with
>> absolute paths.
>>
>> The funny thing is: normally this isn't really a problem for
>> reproducibility, because if the library is installed, the absolute
>> paths will be something along the lines of /usr/lib/.../*.o. And
>> binaries built with that path hard-coded are still going to be
>> reproducible across systems. But when building a C library this
>> is an actual issue.
>>
> 
> If you set a -fdebug-prefix-map=. option and it still doesn't apply
> it to those FILE entries, I would guess that is probably a GCC bug.
> You should either file it upstream or submit a patch to them.

So I've finally figured this out, and the problem was that I was
looking in the wrong spot.

gcc behaves correctly and does interpret -fdebug-prefix-map. However,
there are cases where ld will also create FILE entries in the symbol
table of an executable - and ld doesn't appear to support
-fdebug-prefix-map at all.

Take the following (on amd64):

cat > main.c <<EOF
int main() { return 0; }
EOF
cat > test.S <<EOF
foo:
  ret
EOF
gcc -Wall -c -o main.o main.c
gcc -Wall -c -o test.o test.S
gcc -Wall -o test ${PWD}/main.o ${PWD}/test.o
./test
readelf -sW ./test.o | grep FILE
readelf -sW ./main.o | grep FILE
readelf -sW ./test | grep FILE

You see here that test.o doesn't contain a FILE entry in the
symbol table, but once it's mangled through the linker there is
one now.

If you define the 'foo' symbol in test.S to be global, by adding
.global foo
to that file, the FILE entry disappears from the symbol table.

If you drop the ${PWD}/, then the FILE entry will be relative.

Also, if you use gold (-fuse-ld=gold in the linker line), this
problem doesn't appear either.

Also, if you put the object file into a static library, the
FILE entry is not emitted either:

ar cr test.a ${PWD}/test.o
gcc -Wall -o test ${PWD}/main.o ${PWD}/test.a

The problem also goes away if I add something like
.file "test.S"
to the top of the assembly file to explicitly create a FILE
entry already in the object file.

I believe the issue stems from this code in binutils/bfd:

http://sources.debian.net/src/binutils/2.27.90.20170218-1/bfd/elflink.c/#L9757-L9778

The file name at this point is just the argument passed to the linker,
irrespective of whether it's an absolute path or a relative one, and
no prefix is applied at all.

The initial problem with dietlibc is that on ARM paltforms the
startup code has local symbols in the assembly, which it doesn't
on x86, so the problem has never surfaced on x86 in the past.

Which means this is a broader class of bugs that every time one
has an assembly file that doesn't contain a ".file" directive
and contains local symbols, ld.bfd will output a FILE entry
which may contain an absolute path of the object file (depending on
how the file name was passed to the linker), which can break
reproducibility.


For dietlibc (after Stretch), I can just add .file directives
everywhere, so I can work around this issue.

But to make sure this doesn't happen at all anymore (there might
be other software with hand-crafted assembly out there that has
similar issues), I believe binutils should also be altered to
support SOURCE_PREFIX_MAP in these cases.

Unfortunately, I know way too little about the details of binutils
to feel comfortable in creating a patch (it was hard enough to find
the suspected place where the path is injected), so all I can do 
here is report my findings. Hopefully someone with more binutils-fu
is able to pick this up.

Regards,
Christian