hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Chauhan <hashut...@apache.org>
Subject Re: [discuss] jdk8 support
Date Fri, 03 Jun 2016 05:02:10 GMT
Now that branch-2.1 has been cut, I think its an opportune time to drop
support of jdk7 altogether in master. That is we compile using jdk8 as
well. What do others think?

Ashutosh

On Fri, May 27, 2016 at 7:41 PM, Sergio Pena <sergio.pena@cloudera.com>
wrote:

> I did the change to JDK8 on Ptest now.
>
> Please let me tknow if there are issues with Java8.
>
> - Sergio
>
> On Fri, May 27, 2016 at 2:45 PM, Sergio Pena <sergio.pena@cloudera.com>
> wrote:
>
> > Thanks Mohit.
> >
> > I will plan to do the JDK8 change on Jenkins today EOD and monitor any
> > issues through the weekend.
> >
> > - Sergio
> >
> >
> > On Fri, May 27, 2016 at 2:58 AM, Mohit Sabharwal <mohit@cloudera.com>
> > wrote:
> >
> >> Update on moving Hive2 tests to JDK8: I've addressed almost all test
> >> failures in HIVE-13547 on java8 branch. There is one remaining
> >> open item (HIVE-13834) which is currently assigned. Given current
> >> state of flaky test runs, there might be few more.
> >>
> >> I will work with Sergio to merge the test fixes to master and switch
> >> the Hive2 pre-commit job to use JDK8, hopefully sometime tomorrow.
> >>
> >> After Hive2 tests switch, if your patch sees ordering related test
> >> failures in pre-commit run, it's likely JDK version related and you'll
> >> need to build & re-run the test using JDK8. Number of such tests
> >> should be relatively small.
> >>
> >> On Tue, Apr 19, 2016 at 10:43 AM, Mohit Sabharwal <mohit@cloudera.com>
> >> wrote:
> >>
> >> > Created HIVE-13547 to track switching 2x tests to JDK8.
> >> >
> >> > On Wed, Apr 13, 2016 at 10:02 AM, Sergio Pena <
> sergio.pena@cloudera.com
> >> >
> >> > wrote:
> >> >
> >> >> I agree with such change as JDK7 is not longer supported.
> >> >>
> >> >> Changes on Jenkins and Hive PTest shouldn't be hard. We just need to
> >> >> replace the path from java7 to java8. But I think we should fix all
> >> JDK8
> >> >> issues or most of them before doing the change or we will end up
> >> having a
> >> >> lot of failures on all JIRAs running pre-commit tests.
> >> >>
> >> >> +1 with the change.
> >> >>
> >> >> On Mon, Apr 11, 2016 at 9:34 PM, Siddharth Seth <sseth@apache.org>
> >> wrote:
> >> >>
> >> >> > Option 3 sounds good. I'd ideally like to make JDK8 the minimum
> >> >> requirement
> >> >> > soon as well.
> >> >> >
> >> >> > On Mon, Apr 11, 2016 at 4:59 PM, Szehon Ho <szehon@cloudera.com>
> >> wrote:
> >> >> >
> >> >> > > Sounds like a good plan, +1
> >> >> > >
> >> >> > > On Mon, Apr 11, 2016 at 4:31 PM, Mohit Sabharwal <
> >> mohit@cloudera.com>
> >> >> > > wrote:
> >> >> > >
> >> >> > > > Thanks, Ashutosh. Makes sense to keep the source and
target as
> >> 1.7
> >> >> > since
> >> >> > > > we're not using any JDK8 specific features yet. So,
option (3)
> >> >> > > essentially
> >> >> > > > just means using JDK8 exclusively to build & test
Hive2.
> >> >> > > >
> >> >> > > > On Sat, Apr 9, 2016 at 12:23 PM, Ashutosh Chauhan <
> >> >> > hashutosh@apache.org>
> >> >> > > > wrote:
> >> >> > > >
> >> >> > > > > Hi Mohit,
> >> >> > > > >
> >> >> > > > > I also think option 3 makes sense. We should strive
to keep
> >> test
> >> >> > matrix
> >> >> > > > > small so that we can do fast QA runs for dev patches.
> >> >> > > > > We can just use jdk7 to build & test Hive1
and jdk8 to build
> &
> >> >> test
> >> >> > > > Hive2.
> >> >> > > > > However, I am not sure of explicitly dropping support
> >> altogether
> >> >> for
> >> >> > > jdk7
> >> >> > > > > on Hive2. We should make sure that in pom.xml java
source &
> >> target
> >> >> > > > > compatibility is still 1.7 (which already is the
case
> >> currently)
> >> >> so
> >> >> > > that
> >> >> > > > > Hive2 is still compatible with jdk7. Unit tests
as I said we
> >> can
> >> >> run
> >> >> > on
> >> >> > > > > jdk8.
> >> >> > > > >
> >> >> > > > > Thanks,
> >> >> > > > > Ashutosh
> >> >> > > > >
> >> >> > > > > On Fri, Apr 8, 2016 at 4:34 PM, Mohit Sabharwal
<
> >> >> mohit@cloudera.com>
> >> >> > > > > wrote:
> >> >> > > > >
> >> >> > > > > > Hi folks,
> >> >> > > > > >
> >> >> > > > > > Oracle EOL'ed (ended public updates) for Java
7 in April
> >> 2015.
> >> >> > > > > > In order to support Java 8, we fixed quite
a bunch of tests
> >> in
> >> >> > > > > > HIVE-8607 (*) early last year. However, since
our
> pre-commit
> >> >> tests
> >> >> > > run
> >> >> > > > > > on JDK7 only, any JDK8 test failures are getting
ignored.
> As
> >> a
> >> >> > > result,
> >> >> > > > > > the count has crept up
> >> >> > > > > > <
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/
> >> >> > > > > > >
> >> >> > > > > > back
> >> >> > > > > > from zero to 125.
> >> >> > > > > >
> >> >> > > > > > Some options to address this:
> >> >> > > > > >
> >> >> > > > > > (1) Run pre-commit tests on both JDK7 and
JDK8, in both 1.x
> >> >> > > > > > and 2.x: This will further slow down the pre-commit
run.
> >> >> > > > > >
> >> >> > > > > > (2) Alternate pre-commit test runs between
JDK7 and JDK8
> (in
> >> >> > > > > > both 1.x and 2.x): It's a cheap hack. More
messy, though
> >> >> failures
> >> >> > > > > > won't get ignored.
> >> >> > > > > >
> >> >> > > > > > (3) Drop support for JDK7 in Hive 2.x, i.e.
build and test
> >> Hive
> >> >> 2.x
> >> >> > > on
> >> >> > > > > > JDK8 only. For Hive 1.x, continue running
JDK7 pre-commit
> >> tests.
> >> >> > > > > > The pros are:
> >> >> > > > > > - Simple test matrix.
> >> >> > > > > > - No need to slow down test run or maintain
version
> specific
> >> >> golden
> >> >> > > > files
> >> >> > > > > > (**).
> >> >> > > > > > - 2.x looks like the logical place to move
to JDK8.
> >> >> > > > > > - Users transitioning to JDK8 for all other
services do not
> >> >> have to
> >> >> > > > > > maintain
> >> >> > > > > > multiple java versions on the cluster.
> >> >> > > > > >
> >> >> > > > > > Option (3) looks most attractive to me.
> >> >> > > > > >
> >> >> > > > > > Moving to JDK8 also lines us up better for
Java 9, which is
> >> on
> >> >> the
> >> >> > > > > > horizon (Oracle will end public updates for
Java 8 in Sep
> >> 2017)
> >> >> > > > > >
> >> >> > > > > > Around 100 of the latest crop of failures
are due to one
> >> cause
> >> >> > > > > > (HIVE-13409).
> >> >> > > > > > I can take a pass at triaging the rest if
there is
> consensus
> >> >> around
> >> >> > > > what
> >> >> > > > > > to do overall.
> >> >> > > > > >
> >> >> > > > > > Thanks,
> >> >> > > > > > Mohit
> >> >> > > > > >
> >> >> > > > > > (*) Most test failures are due to JDK8 using
a different
> hash
> >> >> > > function
> >> >> > > > > for
> >> >> > > > > > HashMap  compared to JDK7. This results in
(mostly benign,
> >> but
> >> >> hard
> >> >> > > > > > to debug) ordering differences in q-file output
related to
> >> >> > > > serialization
> >> >> > > > > > order of map entries, numbering of stages
in query plan,
> etc.
> >> >> > > > > >
> >> >> > > > > > (**) In some cases, hash function related
ordering
> >> differences
> >> >> > > > originate
> >> >> > > > > > in external libraries like Avro, antlr, json
,Thrift's map,
> >> etc.
> >> >> > for
> >> >> > > > > which
> >> >> > > > > > code
> >> >> > > > > > changes are either more involved or led to
more test
> >> failures.
> >> >> To
> >> >> > > > > address
> >> >> > > > > > this,
> >> >> > > > > > we added support for version specific golden
files
> >> (HIVE-9109).
> >> >> > > > > Currently,
> >> >> > > > > > there
> >> >> > > > > > are ~40 golden files with ".java1.8.out" extension.
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message