hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Chauhan <hashut...@apache.org>
Subject Re: [discuss] jdk8 support
Date Sat, 09 Apr 2016 19:23:50 GMT
Hi Mohit,

I also think option 3 makes sense. We should strive to keep test matrix
small so that we can do fast QA runs for dev patches.
We can just use jdk7 to build & test Hive1 and jdk8 to build & test Hive2.
However, I am not sure of explicitly dropping support altogether for jdk7
on Hive2. We should make sure that in pom.xml java source & target
compatibility is still 1.7 (which already is the case currently) so that
Hive2 is still compatible with jdk7. Unit tests as I said we can run on
jdk8.

Thanks,
Ashutosh

On Fri, Apr 8, 2016 at 4:34 PM, Mohit Sabharwal <mohit@cloudera.com> wrote:

> Hi folks,
>
> Oracle EOL'ed (ended public updates) for Java 7 in April 2015.
> In order to support Java 8, we fixed quite a bunch of tests in
> HIVE-8607 (*) early last year. However, since our pre-commit tests run
> on JDK7 only, any JDK8 test failures are getting ignored. As a result,
> the count has crept up
> <
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/
> >
> back
> from zero to 125.
>
> Some options to address this:
>
> (1) Run pre-commit tests on both JDK7 and JDK8, in both 1.x
> and 2.x: This will further slow down the pre-commit run.
>
> (2) Alternate pre-commit test runs between JDK7 and JDK8 (in
> both 1.x and 2.x): It's a cheap hack. More messy, though failures
> won't get ignored.
>
> (3) Drop support for JDK7 in Hive 2.x, i.e. build and test Hive 2.x on
> JDK8 only. For Hive 1.x, continue running JDK7 pre-commit tests.
> The pros are:
> - Simple test matrix.
> - No need to slow down test run or maintain version specific golden files
> (**).
> - 2.x looks like the logical place to move to JDK8.
> - Users transitioning to JDK8 for all other services do not have to
> maintain
> multiple java versions on the cluster.
>
> Option (3) looks most attractive to me.
>
> Moving to JDK8 also lines us up better for Java 9, which is on the
> horizon (Oracle will end public updates for Java 8 in Sep 2017)
>
> Around 100 of the latest crop of failures are due to one cause
> (HIVE-13409).
> I can take a pass at triaging the rest if there is consensus around what
> to do overall.
>
> Thanks,
> Mohit
>
> (*) Most test failures are due to JDK8 using a different hash function for
> HashMap  compared to JDK7. This results in (mostly benign, but hard
> to debug) ordering differences in q-file output related to serialization
> order of map entries, numbering of stages in query plan, etc.
>
> (**) In some cases, hash function related ordering differences originate
> in external libraries like Avro, antlr, json ,Thrift's map, etc. for which
> code
> changes are either more involved or led to more test failures.  To address
> this,
> we added support for version specific golden files (HIVE-9109). Currently,
> there
> are ~40 golden files with ".java1.8.out" extension.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message