llvm-toolchain-3.8 on lower arm targets, specifically Debian armel and Raspbian.

[peter green]

llvm-toolchain-3.8 seems to have problems on debian armel and raspbian.

On raspbian it builds but our armv7 contamination checker blocked it 
from entering the repo. Further investigation showed that "compiler-rt" 
was being built with -march=armv7 . I was able to remove the -march with 
some build-system hacker but then I got a failure on
projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S

> [  8%] Building ASM object 
> projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/arm/sync_fetch_and_add_4.S.o
> cd 
> /llvm-toolchain-3.8-3.8/build-llvm/projects/compiler-rt/lib/builtins 
> && /usr/bin/gcc-5  -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS 
> -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS 
> -I/llvm-toolchain-3.8-3.8/build-llvm/projects/compiler-rt/lib/builtins 
> -I/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins 
> -I/llvm-toolchain-3.8-3.8/build-llvm/include 
> -I/llvm-toolchain-3.8-3.8/include  -fPIC -O2 -g -DNDEBUG    
> -mfloat-abi=hard -o 
> CMakeFiles/clang_rt.builtins-armhf.dir/arm/sync_fetch_and_add_4.S.o -c 
> /llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S
> /llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S: 
> Assembler messages:
> /llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20: 
> Error: selected processor does not support `dmb' in Thumb mode
> /llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20: 
> Error: selected processor does not support `ldrex r0,[r12]' in Thumb mode
> /llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20: 
> Error: cannot honor width suffix -- `add r2,r0,r1'
> /llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20: 
> Error: selected processor does not support `strex r3,r2,[r12]' in 
> Thumb mode
> /llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/builtins/arm/sync_fetch_and_add_4.S:20: 
> Error: selected processor does not support `dmb' in Thumb mode
> projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/build.make:1097: 
> recipe for target 
> 'projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/arm/sync_fetch_and_add_4.S.o' 
> failed
> make[4]: *** 
> [projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/arm/sync_fetch_and_add_4.S.o] 
> Error 1
> make[4]: Leaving directory '/llvm-toolchain-3.8-3.8/build-llvm'
> CMakeFiles/Makefile2:12146: recipe for target 
> 'projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/all' 
> failed
> make[3]: *** 
> [projects/compiler-rt/lib/builtins/CMakeFiles/clang_rt.builtins-armhf.dir/all] 
> Error 2
> make[3]: *** Waiting for unfinished jobs...
I traced this back to code in compiler-rt/lib/builtins/arm/sync-ops.h . 
The comments talk about both arm mode and thumb2 mode code but I only 
actually see thumb2 code in the file. I'm kinda stuck here.

I expect this would be a problem for armel too (though it may go 
unnoticed due to the fact the armel autobuilders now run on armv7 
hardware and afaict Debian armel unlike raspbian doesn't have any 
contamination checking in place) but the build doesn't even get that 
far. It fails earlier with.

> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h: In member function 'std::shared_future<void>  llvm::ThreadPool::async(Function&&, Args&&  ...)':
> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h:77:77: error: return type 'class std::shared_future<void>' is incomplete
>     inline std::shared_future<VoidTy>  async(Function&&F, Args&&... ArgList) {
>                                                                               ^
> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h: In member function 'std::shared_future<void>  llvm::ThreadPool::async(Function&&)':
> /«PKGBUILDDIR»/include/llvm/Support/ThreadPool.h:95:57: error: return type 'class std::shared_future<void>' is incomplete
>     inline std::shared_future<VoidTy>  async(Function&&F) {