hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Sabharwal <mo...@cloudera.com>
Subject Re: [discuss] jdk8 support
Date Tue, 19 Apr 2016 17:43:07 GMT
Created HIVE-13547 to track switching 2x tests to JDK8.

On Wed, Apr 13, 2016 at 10:02 AM, Sergio Pena <sergio.pena@cloudera.com>
wrote:

> I agree with such change as JDK7 is not longer supported.
>
> Changes on Jenkins and Hive PTest shouldn't be hard. We just need to
> replace the path from java7 to java8. But I think we should fix all JDK8
> issues or most of them before doing the change or we will end up having a
> lot of failures on all JIRAs running pre-commit tests.
>
> +1 with the change.
>
> On Mon, Apr 11, 2016 at 9:34 PM, Siddharth Seth <sseth@apache.org> wrote:
>
> > Option 3 sounds good. I'd ideally like to make JDK8 the minimum
> requirement
> > soon as well.
> >
> > On Mon, Apr 11, 2016 at 4:59 PM, Szehon Ho <szehon@cloudera.com> wrote:
> >
> > > Sounds like a good plan, +1
> > >
> > > On Mon, Apr 11, 2016 at 4:31 PM, Mohit Sabharwal <mohit@cloudera.com>
> > > wrote:
> > >
> > > > Thanks, Ashutosh. Makes sense to keep the source and target as 1.7
> > since
> > > > we're not using any JDK8 specific features yet. So, option (3)
> > > essentially
> > > > just means using JDK8 exclusively to build & test Hive2.
> > > >
> > > > On Sat, Apr 9, 2016 at 12:23 PM, Ashutosh Chauhan <
> > hashutosh@apache.org>
> > > > wrote:
> > > >
> > > > > Hi Mohit,
> > > > >
> > > > > I also think option 3 makes sense. We should strive to keep test
> > matrix
> > > > > small so that we can do fast QA runs for dev patches.
> > > > > We can just use jdk7 to build & test Hive1 and jdk8 to build
& test
> > > > Hive2.
> > > > > However, I am not sure of explicitly dropping support altogether
> for
> > > jdk7
> > > > > on Hive2. We should make sure that in pom.xml java source & target
> > > > > compatibility is still 1.7 (which already is the case currently)
so
> > > that
> > > > > Hive2 is still compatible with jdk7. Unit tests as I said we can
> run
> > on
> > > > > jdk8.
> > > > >
> > > > > Thanks,
> > > > > Ashutosh
> > > > >
> > > > > On Fri, Apr 8, 2016 at 4:34 PM, Mohit Sabharwal <
> mohit@cloudera.com>
> > > > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > Oracle EOL'ed (ended public updates) for Java 7 in April 2015.
> > > > > > In order to support Java 8, we fixed quite a bunch of tests
in
> > > > > > HIVE-8607 (*) early last year. However, since our pre-commit
> tests
> > > run
> > > > > > on JDK7 only, any JDK8 test failures are getting ignored. As
a
> > > result,
> > > > > > the count has crept up
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/
> > > > > > >
> > > > > > back
> > > > > > from zero to 125.
> > > > > >
> > > > > > Some options to address this:
> > > > > >
> > > > > > (1) Run pre-commit tests on both JDK7 and JDK8, in both 1.x
> > > > > > and 2.x: This will further slow down the pre-commit run.
> > > > > >
> > > > > > (2) Alternate pre-commit test runs between JDK7 and JDK8 (in
> > > > > > both 1.x and 2.x): It's a cheap hack. More messy, though failures
> > > > > > won't get ignored.
> > > > > >
> > > > > > (3) Drop support for JDK7 in Hive 2.x, i.e. build and test Hive
> 2.x
> > > on
> > > > > > JDK8 only. For Hive 1.x, continue running JDK7 pre-commit tests.
> > > > > > The pros are:
> > > > > > - Simple test matrix.
> > > > > > - No need to slow down test run or maintain version specific
> golden
> > > > files
> > > > > > (**).
> > > > > > - 2.x looks like the logical place to move to JDK8.
> > > > > > - Users transitioning to JDK8 for all other services do not
have
> to
> > > > > > maintain
> > > > > > multiple java versions on the cluster.
> > > > > >
> > > > > > Option (3) looks most attractive to me.
> > > > > >
> > > > > > Moving to JDK8 also lines us up better for Java 9, which is
on
> the
> > > > > > horizon (Oracle will end public updates for Java 8 in Sep 2017)
> > > > > >
> > > > > > Around 100 of the latest crop of failures are due to one cause
> > > > > > (HIVE-13409).
> > > > > > I can take a pass at triaging the rest if there is consensus
> around
> > > > what
> > > > > > to do overall.
> > > > > >
> > > > > > Thanks,
> > > > > > Mohit
> > > > > >
> > > > > > (*) Most test failures are due to JDK8 using a different hash
> > > function
> > > > > for
> > > > > > HashMap  compared to JDK7. This results in (mostly benign, but
> hard
> > > > > > to debug) ordering differences in q-file output related to
> > > > serialization
> > > > > > order of map entries, numbering of stages in query plan, etc.
> > > > > >
> > > > > > (**) In some cases, hash function related ordering differences
> > > > originate
> > > > > > in external libraries like Avro, antlr, json ,Thrift's map,
etc.
> > for
> > > > > which
> > > > > > code
> > > > > > changes are either more involved or led to more test failures.
> To
> > > > > address
> > > > > > this,
> > > > > > we added support for version specific golden files (HIVE-9109).
> > > > > Currently,
> > > > > > there
> > > > > > are ~40 golden files with ".java1.8.out" extension.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message