hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Sabharwal <mo...@cloudera.com>
Subject Re: [discuss] jdk8 support
Date Mon, 11 Apr 2016 23:31:09 GMT
Thanks, Ashutosh. Makes sense to keep the source and target as 1.7 since
we're not using any JDK8 specific features yet. So, option (3) essentially
just means using JDK8 exclusively to build & test Hive2.

On Sat, Apr 9, 2016 at 12:23 PM, Ashutosh Chauhan <hashutosh@apache.org>
wrote:

> Hi Mohit,
>
> I also think option 3 makes sense. We should strive to keep test matrix
> small so that we can do fast QA runs for dev patches.
> We can just use jdk7 to build & test Hive1 and jdk8 to build & test Hive2.
> However, I am not sure of explicitly dropping support altogether for jdk7
> on Hive2. We should make sure that in pom.xml java source & target
> compatibility is still 1.7 (which already is the case currently) so that
> Hive2 is still compatible with jdk7. Unit tests as I said we can run on
> jdk8.
>
> Thanks,
> Ashutosh
>
> On Fri, Apr 8, 2016 at 4:34 PM, Mohit Sabharwal <mohit@cloudera.com>
> wrote:
>
> > Hi folks,
> >
> > Oracle EOL'ed (ended public updates) for Java 7 in April 2015.
> > In order to support Java 8, we fixed quite a bunch of tests in
> > HIVE-8607 (*) early last year. However, since our pre-commit tests run
> > on JDK7 only, any JDK8 test failures are getting ignored. As a result,
> > the count has crept up
> > <
> >
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/
> > >
> > back
> > from zero to 125.
> >
> > Some options to address this:
> >
> > (1) Run pre-commit tests on both JDK7 and JDK8, in both 1.x
> > and 2.x: This will further slow down the pre-commit run.
> >
> > (2) Alternate pre-commit test runs between JDK7 and JDK8 (in
> > both 1.x and 2.x): It's a cheap hack. More messy, though failures
> > won't get ignored.
> >
> > (3) Drop support for JDK7 in Hive 2.x, i.e. build and test Hive 2.x on
> > JDK8 only. For Hive 1.x, continue running JDK7 pre-commit tests.
> > The pros are:
> > - Simple test matrix.
> > - No need to slow down test run or maintain version specific golden files
> > (**).
> > - 2.x looks like the logical place to move to JDK8.
> > - Users transitioning to JDK8 for all other services do not have to
> > maintain
> > multiple java versions on the cluster.
> >
> > Option (3) looks most attractive to me.
> >
> > Moving to JDK8 also lines us up better for Java 9, which is on the
> > horizon (Oracle will end public updates for Java 8 in Sep 2017)
> >
> > Around 100 of the latest crop of failures are due to one cause
> > (HIVE-13409).
> > I can take a pass at triaging the rest if there is consensus around what
> > to do overall.
> >
> > Thanks,
> > Mohit
> >
> > (*) Most test failures are due to JDK8 using a different hash function
> for
> > HashMap  compared to JDK7. This results in (mostly benign, but hard
> > to debug) ordering differences in q-file output related to serialization
> > order of map entries, numbering of stages in query plan, etc.
> >
> > (**) In some cases, hash function related ordering differences originate
> > in external libraries like Avro, antlr, json ,Thrift's map, etc. for
> which
> > code
> > changes are either more involved or led to more test failures.  To
> address
> > this,
> > we added support for version specific golden files (HIVE-9109).
> Currently,
> > there
> > are ~40 golden files with ".java1.8.out" extension.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message