Mailing-List: contact dev-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@impala.incubator.apache.org
Message-Id: <201604232205.u3NM5hee022868@ip-10-146-233-104.ec2.internal>
Date: Sat, 23 Apr 2016 22:05:43 +0000
From: "Internal Jenkins (Code Review)" <gerrit@cloudera.org>
To: Tim Armstrong <tarmstrong@cloudera.com>, impala-cr@cloudera.com,
        dev@impala.incubator.apache.org
Subject: 
 =?UTF-8?Q?[Impala-CR](cdh5-trunk)_IMPALA-775=2CIMPALA-3374:_Upgrade_LLVM_to_3.8.0=0A?=
In-Reply-To: 
 <gerrit.1457463056000.I17d7afd05ad3b472a0bfe035bfc3daada5597b2d@gerrit.cloudera.org>
References: 
 <gerrit.1457463056000.I17d7afd05ad3b472a0bfe035bfc3daada5597b2d@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.10-rc0

Internal Jenkins has submitted this change and it was merged.

Change subject: IMPALA-775,IMPALA-3374: Upgrade LLVM to 3.8.0
......................................................................


IMPALA-775,IMPALA-3374: Upgrade LLVM to 3.8.0

This requires various changes for Impala to be fully functional with the
new version of LLVM.

The original JIT was removed from LLVM, we need to switch to the new
MCJIT API and implementation.

MCJIT only supports module-at-a-time compilation, so the module must
be finalised before any compilation happens. We did't depend on the
old behaviour deeply, but various small fixes were required.

MCJIT requires that every IR module has a name.

We relied on the old JIT's workaround for the __dso_handle symbol,
which we have to emulate for MCJIT with a customer memory manager
until we can get rid of global initialisers in cross-compiled code.

LLVM made a number of incompatible API changes and reorganised headers.

Clang took over responsibility for padding structs by marking structs
as packed and inserting bytes so that members are aligned correctly
(previously it relies LLVM aligning struct members based on the
target's alignment rules). This means Impala also needs to manually
pad its structs since clang-emitted structs look to LLVM like they have
do not need to be inlined.

Our inlining pass would require some modification to work and is
redundant with LLVM's inlining pass, so was removed along with the
unused subexpr elimination pass.

LLVM now depends on another system library libtinfo, so we use
llvm-config to get the required system libs directly.

There were various issues with __builtin_add_overflow and
__builtin_mul_overflow that are newly available in LLVM 3.8.
First, LLVM emitted a call to a function in libclang_rt, which
we don't link in and has symbols that conflict with
the gcc runtime library. Second, the performance actually regressed
by using the builtins (I tested this manually by copying across the
definition of the required function).

Change-Id: I17d7afd05ad3b472a0bfe035bfc3daada5597b2d
Reviewed-on: http://gerrit.cloudera.org:8080/2486
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
Tested-by: Internal Jenkins
---
M be/CMakeLists.txt
M be/src/benchmarks/hash-benchmark.cc
M be/src/benchmarks/overflow-benchmark.cc
M be/src/codegen/CMakeLists.txt
M be/src/codegen/codegen-anyval.cc
M be/src/codegen/instruction-counter-test.cc
M be/src/codegen/llvm-codegen-test.cc
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
A be/src/codegen/mcjit-mem-mgr.h
D be/src/codegen/subexpr-elimination.cc
D be/src/codegen/subexpr-elimination.h
M be/src/exec/aggregation-node.cc
M be/src/exec/hash-join-node.cc
M be/src/exec/hash-table.cc
M be/src/exec/hdfs-avro-scanner.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/old-hash-table.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/partitioned-hash-join-node.cc
M be/src/exec/text-converter.cc
M be/src/exprs/expr-codegen-test.cc
M be/src/exprs/expr.cc
M be/src/exprs/scalar-fn-call.cc
M be/src/exprs/slot-ref.cc
M be/src/runtime/decimal-value.inline.h
M be/src/runtime/descriptors.cc
M be/src/runtime/descriptors.h
M be/src/runtime/lib-cache.cc
M be/src/runtime/runtime-state.cc
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M cmake_modules/FindLlvm.cmake
33 files changed, 545 insertions(+), 757 deletions(-)

Approvals:
  Internal Jenkins: Verified
  Tim Armstrong: Looks good to me, approved


-- 
To view, visit http://gerrit.cloudera.org:8080/2486
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I17d7afd05ad3b472a0bfe035bfc3daada5597b2d
Gerrit-PatchSet: 16
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <skye@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>