hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xuefu Zhang <xzh...@cloudera.com>
Subject Re: [DISCUSS] Do we still support hadoop-1.2.x API (-Phadoop-1)?
Date Sat, 02 May 2015 00:37:48 GMT
To close the loop, I have created HIVE-10597 to fix all the compilation
errors on Hadoop 1. Once this is in, @Szehon, could you please include
compilation on Hadoop-1 in precommit test?

Whether or when to remove Hadoop 1 deserves a formal [DISCUSS] thread.

Thanks,
Xuefu

On Thu, Apr 30, 2015 at 12:53 PM, Sergey Shelukhin <sergey@hortonworks.com>
wrote:

> That runs the risk of not testing the code being committed against the
> Hadoop version for which it¹s actually going to be used.
> I¹d be +1 on deprecating Hadoop 1 support in a dot version (1.3?). It will
> also provide impetus for people to upgrade.
>
> It will allow us to remove some shims tooŠ
>
>
> On 15/4/29, 12:13, "Xuefu Zhang" <xzhang@cloudera.com> wrote:
>
> >I think whether and when to deprecate Hadoop-1 is better to be moved to an
> >official [DISCUSS] thread. What I disliked is the sentiment that I felt
> >when we decide to deprecate something just because it's unfortunately
> >broken for one reason or the other. Nevertheless, we should fix these bugs
> >at this moment. We fixed such issues in previous release as well.
> >
> >I concur that we should have caught these breakages before they made their
> >way into the code. Previously we discussed about running pre-commit tests
> >alternatively or even randomly with either Hadoop-1 or Hadoop-2 profile.
> >Now it seems critical to do so. Thus, I support Szehon's proposal to make
> >this happen. This is orthogonal to the deprecation discussion.
> >
> >Thanks,
> >Xuefu
> >
> >On Wed, Apr 29, 2015 at 11:30 AM, Szehon Ho <szehon@cloudera.com> wrote:
> >
> >> Are you
> >> > personally willing to do the work to make sure that Hive works with
> >> Hadoop
> >> > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443, 10444)
> >> > because
> >> > trunk currently does not work with Hadoop 1.x and no one noticed.
> >>
> >> We setup a Hadoop-1 build sometime back and made sure a lot of breakage
> >> were fixed, but had not hooked it up to HiveQA (precommit) due to
> >>concerns
> >> of the time it would take to run these tests.  It was a balance between
> >> getting faster HiveQA results for everyone and testing all the paths,
> >>even
> >> today there are many paths we don't test.
> >>
> >> So if these JIRA's are fixed, would folks be amenable to alternate
> >>testing
> >> hadoop-1 and hadoop-2 on HiveQA precommit runs and for devs to not
> >>commit
> >> further breakages of Hadoop 1.x compat, for another release?  Yes I do
> >> understand its a burden to maintain hadoop-1.
> >>
> >> if you use HDP, CDH or other hadoop distro then hadoop-1.x and hive-1.2
> >> > combination does not exist.
> >> >
> >>
> >> I think Apache is agnostic to specific vendors so should not discuss
> >>based
> >> on that, but vendors still have this combination, I know of users that
> >>use
> >> Hadoop-1 and latest Hive 1.1.
> >>
> >> Thanks,
> >> Szehon
> >>
> >> On Wed, Apr 29, 2015 at 11:07 AM, Alexander Pivovarov <
> >> apivovarov@gmail.com>
> >> wrote:
> >>
> >> > HDP and CDH distro are based on hadoop-2.x since 2013-2014
> >> >
> >> > if you use HDP, CDH or other hadoop distro then hadoop-1.x and
> >>hive-1.2
> >> > combination does not exist.
> >> > hdp-1.3 has hive-0.11
> >> > cdh-4.7 has hive-0.10
> >> >
> >> > The only way to install hive-1.2 on hadoop-1.x is to do it manually.
> >> > But if you manually install hadoop components then most probably you
> >>will
> >> > install hadoop-2.x
> >> >
> >> > So, we can say that hadoop-1.x and hive-1.2 combination should not
> >>exist
> >> in
> >> > real life scenario.
> >> >
> >> >
> >> >
> >> > On Wed, Apr 29, 2015 at 8:03 AM, Owen O'Malley <omalley@apache.org>
> >> wrote:
> >> >
> >> > > On Tue, Apr 28, 2015 at 10:16 PM, Szehon Ho <szehon@cloudera.com>
> >> wrote:
> >> > >
> >> > > > Yes, question is whether users want to run latest Hive version
on
> >> > Hadoop
> >> > > > 1.x clusters.
> >> > > >
> >> > >
> >> > > But you ignored the more important question that I asked. Are you
> >> > > personally willing to do the work to make sure that Hive works with
> >> > Hadoop
> >> > > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443,
> >>10444)
> >> > > because
> >> > > trunk currently does not work with Hadoop 1.x and no one noticed.
> >> > >
> >> > > Supporting Hadoop 1.x costs the Hive project a lot of work and
> >>provides
> >> > > very little in return.
> >> > >
> >> > > .. Owen
> >> > >
> >> >
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message