hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Pivovarov <apivova...@gmail.com>
Subject Re: [DISCUSS] Do we still support hadoop-1.2.x API (-Phadoop-1)?
Date Wed, 29 Apr 2015 21:08:28 GMT
Szehon, can we agree that hive-1.2 is the last hive version which supports
hadoop-1.
Starting from hive-1.3 it requires hadoop-2

So, most probably "organizations" will start to install hive-1.2 in the
next 6 months.
Then they will use it for 1-2 years until they want to upgrade to the next
hive version.
So, for the next 2 years "organizations" as good. They can use hive-1.2
with hadoop-1.

But after 2 years (in 2017) they are kinda forced to uprate to hadoop-2 if
they want to use hive-1.3

Alex


On Wed, Apr 29, 2015 at 1:51 PM, Szehon Ho <szehon@cloudera.com> wrote:

> Alex, I'd not confuse these two things (hadoop-1 and HiveCLI), after the
> proposed change from hive script to wrap beeline, if we do the job right
> then user should not notice much difference.  There we can eliminate dev
> work while not affecting user.  Whereas migration of an organization from
> using Hadoop-1 cluster to Hadoop-2 cluster is a much-bigger change.
>
> Thanks,
> Szehon
>
>
>
> On Wed, Apr 29, 2015 at 1:27 PM, Thejas Nair <thejas.nair@gmail.com>
> wrote:
>
> > I agree, the number of users who use Hive 1.1 and newer versions
> > against Hadoop 1.x is likely to very small. (Most likely an orders of
> > magnitude smaller than hive cli users in same versions).
> > The big improvements in Hive 1.x line require Hadoop 2.x. Users who
> > are motivated to upgrade to newer 1.x lines are likely to be using
> > Hadoop 2.x .
> >
> >
> >
> > On Wed, Apr 29, 2015 at 12:41 PM, Alexander Pivovarov
> > <apivovarov@gmail.com> wrote:
> > > Xuefu,
> > >
> > > Remember when it was beeline discussion you mentioned "Hive community
> > don't
> > > need to maintain two code paths"
> > > Totally agree. Resources are limited.
> > >
> > > I believe the same approach is applicable here.
> > > Lets assume hadoop-1 + hive-1.2 is going to be used by 1% of the users.
> > >
> > > We'd better spend resources on implementing new functionality (useful
> for
> > > 99% of the users) instead of spending resources to support hadoop-1
> > profile
> > > (useful for 1% of the users).
> > >
> > > Alex
> > >
> > >
> > > On Wed, Apr 29, 2015 at 12:13 PM, Xuefu Zhang <xzhang@cloudera.com>
> > wrote:
> > >
> > >> I think whether and when to deprecate Hadoop-1 is better to be moved
> to
> > an
> > >> official [DISCUSS] thread. What I disliked is the sentiment that I
> felt
> > >> when we decide to deprecate something just because it's unfortunately
> > >> broken for one reason or the other. Nevertheless, we should fix these
> > bugs
> > >> at this moment. We fixed such issues in previous release as well.
> > >>
> > >> I concur that we should have caught these breakages before they made
> > their
> > >> way into the code. Previously we discussed about running pre-commit
> > tests
> > >> alternatively or even randomly with either Hadoop-1 or Hadoop-2
> profile.
> > >> Now it seems critical to do so. Thus, I support Szehon's proposal to
> > make
> > >> this happen. This is orthogonal to the deprecation discussion.
> > >>
> > >> Thanks,
> > >> Xuefu
> > >>
> > >> On Wed, Apr 29, 2015 at 11:30 AM, Szehon Ho <szehon@cloudera.com>
> > wrote:
> > >>
> > >> > Are you
> > >> > > personally willing to do the work to make sure that Hive works
> with
> > >> > Hadoop
> > >> > > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443,
> > 10444)
> > >> > > because
> > >> > > trunk currently does not work with Hadoop 1.x and no one noticed.
> > >> >
> > >> > We setup a Hadoop-1 build sometime back and made sure a lot of
> > breakage
> > >> > were fixed, but had not hooked it up to HiveQA (precommit) due to
> > >> concerns
> > >> > of the time it would take to run these tests.  It was a balance
> > between
> > >> > getting faster HiveQA results for everyone and testing all the
> paths,
> > >> even
> > >> > today there are many paths we don't test.
> > >> >
> > >> > So if these JIRA's are fixed, would folks be amenable to alternate
> > >> testing
> > >> > hadoop-1 and hadoop-2 on HiveQA precommit runs and for devs to not
> > commit
> > >> > further breakages of Hadoop 1.x compat, for another release?  Yes
I
> do
> > >> > understand its a burden to maintain hadoop-1.
> > >> >
> > >> > if you use HDP, CDH or other hadoop distro then hadoop-1.x and
> > hive-1.2
> > >> > > combination does not exist.
> > >> > >
> > >> >
> > >> > I think Apache is agnostic to specific vendors so should not discuss
> > >> based
> > >> > on that, but vendors still have this combination, I know of users
> that
> > >> use
> > >> > Hadoop-1 and latest Hive 1.1.
> > >> >
> > >> > Thanks,
> > >> > Szehon
> > >> >
> > >> > On Wed, Apr 29, 2015 at 11:07 AM, Alexander Pivovarov <
> > >> > apivovarov@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > HDP and CDH distro are based on hadoop-2.x since 2013-2014
> > >> > >
> > >> > > if you use HDP, CDH or other hadoop distro then hadoop-1.x and
> > hive-1.2
> > >> > > combination does not exist.
> > >> > > hdp-1.3 has hive-0.11
> > >> > > cdh-4.7 has hive-0.10
> > >> > >
> > >> > > The only way to install hive-1.2 on hadoop-1.x is to do it
> manually.
> > >> > > But if you manually install hadoop components then most probably
> you
> > >> will
> > >> > > install hadoop-2.x
> > >> > >
> > >> > > So, we can say that hadoop-1.x and hive-1.2 combination should
not
> > >> exist
> > >> > in
> > >> > > real life scenario.
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Wed, Apr 29, 2015 at 8:03 AM, Owen O'Malley <
> omalley@apache.org>
> > >> > wrote:
> > >> > >
> > >> > > > On Tue, Apr 28, 2015 at 10:16 PM, Szehon Ho <
> szehon@cloudera.com>
> > >> > wrote:
> > >> > > >
> > >> > > > > Yes, question is whether users want to run latest Hive
version
> > on
> > >> > > Hadoop
> > >> > > > > 1.x clusters.
> > >> > > > >
> > >> > > >
> > >> > > > But you ignored the more important question that I asked.
Are
> you
> > >> > > > personally willing to do the work to make sure that Hive
works
> > with
> > >> > > Hadoop
> > >> > > > 1.x? There are five new jiras (HIVE-10430, 10431, 10442,
10443,
> > >> 10444)
> > >> > > > because
> > >> > > > trunk currently does not work with Hadoop 1.x and no one
> noticed.
> > >> > > >
> > >> > > > Supporting Hadoop 1.x costs the Hive project a lot of work
and
> > >> provides
> > >> > > > very little in return.
> > >> > > >
> > >> > > > .. Owen
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message