Auto merge of #47828 - alexcrichton:llvm-6, r=<try>

rustc: Upgrade to LLVM 6

The following submodules have been updated for a new version of LLVM:

- `src/llvm`
- `src/libcompiler_builtins` - transitively contains compiler-rt
- `src/dlmalloc`

This also updates the docker container for dist-i686-freebsd as the old 16.04
container is no longer capable of building LLVM. The
compiler-rt/compiler-builtins and dlmalloc updates are pretty routine without
much interesting happening, but the LLVM update here is of particular note.
Unlike previous updates I haven't cherry-picked all existing patches we had on
top of our LLVM branch as we have a [huge amount][patches4] and have at this
point forgotten what most of them are for. Instead I started from the current
`release_60` branch in LLVM and only applied patches that were necessary to get
our tests working and building.

The [current set of custom rustc-specific patches](rust-lang/llvm@f128612...rust-llvm-release-6-0-0) included in this LLVM update are:

* rust-lang/llvm@1187443 - this is how we actually implement
  `cfg(target_feature)` for now and continues to not be upstreamed. While a
  hazard for SIMD stabilization this commit is otherwise keeping the status
  quo of a small rustc-specific feature.
* rust-lang/llvm@013f2ec - this is a rustc-specific optimization that we haven't
  upstreamed, notably teaching LLVM about our allocation-related routines (which
  aren't malloc/free). Once we stabilize the global allocator routines we will
  likely want to upstream this patch, but for now it seems reasonable to keep it
  on our fork.
* rust-lang/llvm@a65bbfd - I found this necessary to fix compilation of LLVM in
  our 32-bit linux container. I'm not really sure why it's necessary but my
  guess is that it's because of the absolutely ancient glibc that we're using.
  In any case it's only updating pieces we're not actually using in LLVM so I'm
  hoping it'll turn out alright. This doesn't seem like something we'll want to
  upstream.c
* rust-lang/llvm@77ab1f0 - this is what's actually enabling LLVM to build in our
  i686-freebsd container, I'm not really sure what's going on but we for sure
  probably don't want to upstream this and otherwise it seems not too bad for
  now at least.
* rust-lang/llvm@9eb9267 - we currently suffer on MSVC from an [upstream bug]
  which although diagnosed to a particular revision isn't currently fixed
  upstream (and the bug itself doesn't seem too active). This commit is a
  partial revert of the suspected cause of this regression (found via a
  bisection). I'm sort of hoping that this eventually gets fixed upstream with a
  similar fix (which we can replace in our branch), but for now I'm also hoping
  it's a relatively harmless change to have.

After applying these patches (plus one [backport] which should be [backported
upstream][llvm-back]) I believe we should have all tests working on all
platforms in our current test suite. I'm like 99% sure that we'll need some more
backports as issues are reported for LLVM 6 when this propagates through
nightlies, but that's sort of just par for the course nowadays!

In any case though some extra scrutiny of the patches here would definitely be
welcome, along with scrutiny of the "missing patches" like a [change to pass
manager order](rust-lang/llvm@2717444), [another change to pass manager
order](rust-lang/llvm@c782feb), some [compile fixes for
sparc](rust-lang/llvm@1a83de6), and some [fixes for
solaris](rust-lang/llvm@c2bfe0a).

[patches4]: rust-lang/llvm@5401fdf...rust-llvm-release-4-0-1
[backport]: rust-lang/llvm@5c54c25
[llvm-back]: https://bugs.llvm.org/show_bug.cgi?id=36114
[upstream bug]: https://bugs.llvm.org/show_bug.cgi?id=36096

---

The update to LLVM 6 is desirable for a number of reasons, notably:

* This'll allow us to keep up with the upstream wasm backend, picking up new
  features as they start landing.
* Upstream LLVM has fixed a number of SIMD-related compilation errors,
  especially around AVX-512 and such.
* There's a few assorted known bugs which are fixed in LLVM 5 and aren't fixed
  in the LLVM 4 branch we're using.
* Overall it's not a great idea to stagnate with our codegen backend!

This update is mostly powered by #47730 which is allowing us to update LLVM
*independent* of the version of LLVM that Emscripten is locked to. This means
that when compiling code for Emscripten we'll still be using the old LLVM 4
backend, but when compiling code for any other target we'll be using the new
LLVM 6 target. Once Emscripten updates we may no longer need this distinction,
but we're not sure when that will happen!

Closes #43370
Closes #43418
Closes #47015
Closes #47683
Closes rust-lang/stdarch#157
Closes rust-lang-nursery/rust-wasm#3

Author: bors

.travis.yml

@@ -13,16 +13,16 @@ matrix:
   include:
     # Images used in testing PR and try-build should be run first.
     - env: IMAGE=x86_64-gnu-llvm-3.9 RUST_BACKTRACE=1
-      if: type = pull_request OR branch = auto
+      if: type = pull_request OR branch = try
 
     - env: IMAGE=dist-x86_64-linux DEPLOY=1
-      if: branch = try OR branch = auto
+      if: branch = try OR branch = try
 
     # "alternate" deployments, these are "nightlies" but have LLVM assertions
     # turned on, they're deployed to a different location primarily for
     # additional testing.
     - env: IMAGE=dist-x86_64-linux DEPLOY_ALT=1
-      if: branch = try OR branch = auto
+      if: branch = try OR branch = try
 
     - env: >
         RUST_CHECK_TARGET=dist
@@ -36,7 +36,7 @@ matrix:
         NO_DEBUG_ASSERTIONS=1
       os: osx
       osx_image: xcode7.3
-      if: branch = auto
+      if: branch = try
 
     # macOS builders. These are placed near the beginning because they are very
     # slow to run.
@@ -57,7 +57,7 @@ matrix:
         NO_DEBUG_ASSERTIONS=1
       os: osx
       osx_image: xcode8.3
-      if: branch = auto
+      if: branch = try
 
     - env: >
         RUST_CHECK_TARGET=check
@@ -71,7 +71,7 @@ matrix:
         NO_DEBUG_ASSERTIONS=1
       os: osx
       osx_image: xcode8.3
-      if: branch = auto
+      if: branch = try
 
     # OSX builders producing releases. These do not run the full test suite and
     # just produce a bunch of artifacts.
@@ -91,7 +91,7 @@ matrix:
         NO_DEBUG_ASSERTIONS=1
       os: osx
       osx_image: xcode7.3
-      if: branch = auto
+      if: branch = try
 
     - env: >
         RUST_CHECK_TARGET=dist
@@ -105,79 +105,79 @@ matrix:
         NO_DEBUG_ASSERTIONS=1
       os: osx
       osx_image: xcode7.3
-      if: branch = auto
+      if: branch = try
 
     # Linux builders, remaining docker images
     - env: IMAGE=arm-android
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=armhf-gnu
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-various-1 DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-various-2 DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-aarch64-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-android DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-arm-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-armhf-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-armv7-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-i586-gnu-i586-i686-musl DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-i686-freebsd DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-i686-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-mips-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-mips64-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-mips64el-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-mipsel-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-powerpc-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-powerpc64-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-powerpc64le-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-s390x-linux DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-x86_64-freebsd DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-x86_64-musl DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=dist-x86_64-netbsd DEPLOY=1
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=asmjs
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=i686-gnu
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=i686-gnu-nopt
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=wasm32-unknown
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=x86_64-gnu
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=x86_64-gnu-full-bootstrap
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=x86_64-gnu-aux
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=x86_64-gnu-tools
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=x86_64-gnu-debug
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=x86_64-gnu-nopt
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=x86_64-gnu-distcheck
-      if: branch = auto
+      if: branch = try
     - env: IMAGE=x86_64-gnu-incremental
-      if: branch = auto
+      if: branch = try
 
     - stage: publish toolstate
       if: branch = master AND type = push

appveyor.yml

@@ -181,7 +181,7 @@ on_failure:
 
 branches:
   only:
-    - auto
+    - try
 
 before_deploy:
   - ps: |

src/bootstrap/native.rs

@@ -186,7 +186,7 @@ impl Step for Llvm {
         }
 
         // http://llvm.org/docs/HowToCrossCompileLLVM.html
-        if target != build.build {
+        if target != build.build && !emscripten {
             builder.ensure(Llvm {
                 target: build.build,
                 emscripten: false,

src/ci/docker/dist-i686-freebsd/Dockerfile

@@ -1,4 +1,4 @@
-FROM ubuntu:16.04
+FROM ubuntu:18.04
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
   clang \

src/ci/run.sh

@@ -23,7 +23,7 @@ fi
 ci_dir=`cd $(dirname $0) && pwd`
 source "$ci_dir/shared.sh"
 
-if [ "$TRAVIS" == "true" ] && [ "$TRAVIS_BRANCH" != "auto" ]; then
+if [ "$TRAVIS" == "true" ] && [ "$TRAVIS_BRANCH" != "try" ]; then
     RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --enable-quiet-tests"
 fi
 

src/dlmalloc

@@ -552,9 +552,11 @@ static unsigned fromRust(LLVMRustDIFlags Flags) {
   if (isSet(Flags & LLVMRustDIFlags::FlagRValueReference)) {
     Result |= DINode::DIFlags::FlagRValueReference;
   }
+#if LLVM_VERSION_LE(4, 0)
   if (isSet(Flags & LLVMRustDIFlags::FlagExternalTypeRef)) {
     Result |= DINode::DIFlags::FlagExternalTypeRef;
   }
+#endif
   if (isSet(Flags & LLVMRustDIFlags::FlagIntroducedVirtual)) {
     Result |= DINode::DIFlags::FlagIntroducedVirtual;
   }

src/libcompiler_builtins

@@ -1,4 +1,4 @@
 # If this file is modified, then llvm will be (optionally) cleaned and then rebuilt.
 # The actual contents of this file do not matter, but to trigger a change on the
 # build bots then the contents should be changed so git updates the mtime.
-2018-01-25
+2018-02-09

src/llvm

@@ -15,11 +15,14 @@
 // Unfortunately, LLVM has no "disable" option for this, so we have to set
 // "enable" to 0 instead.
 
-// compile-flags:-g -Cllvm-args=-enable-tail-merge=0
+// compile-flags:-g -Cllvm-args=-enable-tail-merge=0 -Cllvm-args=-opt-bisect-limit=0
 // ignore-pretty issue #37195
 // ignore-cloudabi spawning processes is not supported
 // ignore-emscripten spawning processes is not supported
 
+// note that above `-opt-bisect-limit=0` is used to basically disable
+// optimizations
+
 use std::env;
 
 #[path = "backtrace-debuginfo-aux.rs"] mod aux;
@@ -114,25 +117,34 @@ fn outer(mut counter: i32, main_pos: Pos) {
     inner_inlined(&mut counter, main_pos, pos!());
 }
 
-fn check_trace(output: &str, error: &str) {
+fn check_trace(output: &str, error: &str) -> Result<(), String> {
     // reverse the position list so we can start with the last item (which was the first line)
     let mut remaining: Vec<&str> = output.lines().map(|s| s.trim()).rev().collect();
 
-    assert!(error.contains("stack backtrace"), "no backtrace in the error: {}", error);
+    if !error.contains("stack backtrace") {
+        return Err(format!("no backtrace found in stderr:\n{}", error))
+    }
     for line in error.lines() {
         if !remaining.is_empty() && line.contains(remaining.last().unwrap()) {
             remaining.pop();
         }
     }
-    assert!(remaining.is_empty(),
-            "trace does not match position list: {}\n---\n{}", error, output);
+    if !remaining.is_empty() {
+        return Err(format!("trace does not match position list\n\
+            still need to find {:?}\n\n\
+            --- stdout\n{}\n\
+            --- stderr\n{}",
+            remaining, output, error))
+    }
+    Ok(())
 }
 
 fn run_test(me: &str) {
     use std::str;
     use std::process::Command;
 
     let mut i = 0;
+    let mut errors = Vec::new();
     loop {
         let out = Command::new(me)
                           .env("RUST_BACKTRACE", "full")
@@ -143,10 +155,20 @@ fn run_test(me: &str) {
             assert!(output.contains("done."), "bad output for successful run: {}", output);
             break;
         } else {
-            check_trace(output, error);
+            if let Err(e) = check_trace(output, error) {
+                errors.push(e);
+            }
         }
         i += 1;
     }
+    if errors.len() > 0 {
+        for error in errors {
+            println!("---------------------------------------");
+            println!("{}", error);
+        }
+
+        panic!("found some errors");
+    }
 }
 
 #[inline(never)]