Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Received-SPF: softfail (athena.apache.org: transitioning domain of
 szehon@cloudera.com does not designate 54.191.145.13 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CA+mdTd3WEcywTVGBpvTAgq89NgJztjGn3nEy_frPQFQmnB+_yA@mail.gmail.com>
References: 
 <CAHfHakFKmMN+-Ov6+4NsbyfYbWbhxF=T_YyidGTXJVP7bf6v8w@mail.gmail.com>
 <CAKKt98Tg4aeD3nh6kkNMFwcB2fRPfEOJTCV7Wr2U+95KWA4Rqw@mail.gmail.com>
 <CAPs2cbGvNKzA64_8Q-2keigbDLHTVzqNOAEKyKBSFZbufjoQGw@mail.gmail.com>
 <CADx-ob2LXAHKZao+H=TeZ9mRr-hWzN68bgZVK7wV3_dST0JFSw@mail.gmail.com>
 <CAKKt98SfjKf6ZkgsKKh8pRD-L680bPCUz+X=b4zB7BW9HFBW=g@mail.gmail.com>
 <CA+mdTd3WEcywTVGBpvTAgq89NgJztjGn3nEy_frPQFQmnB+_yA@mail.gmail.com>
From: Szehon Ho <szehon@cloudera.com>
Date: Wed, 29 Apr 2015 13:51:05 -0700
Message-ID: 
 <CAPs2cbHKbSayNsHMU_8Uc06sfuL6F+cgOKLhbdmoc5KChuNTcA@mail.gmail.com>
Subject: Re: [DISCUSS] Do we still support hadoop-1.2.x API (-Phadoop-1)?
To: "dev@hive.apache.org" <dev@hive.apache.org>
Content-Type: multipart/alternative; boundary=001a1134effa84bc550514e3248e

--001a1134effa84bc550514e3248e
Content-Type: text/plain; charset=UTF-8

Alex, I'd not confuse these two things (hadoop-1 and HiveCLI), after the
proposed change from hive script to wrap beeline, if we do the job right
then user should not notice much difference.  There we can eliminate dev
work while not affecting user.  Whereas migration of an organization from
using Hadoop-1 cluster to Hadoop-2 cluster is a much-bigger change.

Thanks,
Szehon


On Wed, Apr 29, 2015 at 1:27 PM, Thejas Nair <thejas.nair@gmail.com> wrote:

> I agree, the number of users who use Hive 1.1 and newer versions
> against Hadoop 1.x is likely to very small. (Most likely an orders of
> magnitude smaller than hive cli users in same versions).
> The big improvements in Hive 1.x line require Hadoop 2.x. Users who
> are motivated to upgrade to newer 1.x lines are likely to be using
> Hadoop 2.x .
>
>
>
> On Wed, Apr 29, 2015 at 12:41 PM, Alexander Pivovarov
> <apivovarov@gmail.com> wrote:
> > Xuefu,
> >
> > Remember when it was beeline discussion you mentioned "Hive community
> don't
> > need to maintain two code paths"
> > Totally agree. Resources are limited.
> >
> > I believe the same approach is applicable here.
> > Lets assume hadoop-1 + hive-1.2 is going to be used by 1% of the users.
> >
> > We'd better spend resources on implementing new functionality (useful for
> > 99% of the users) instead of spending resources to support hadoop-1
> profile
> > (useful for 1% of the users).
> >
> > Alex
> >
> >
> > On Wed, Apr 29, 2015 at 12:13 PM, Xuefu Zhang <xzhang@cloudera.com>
> wrote:
> >
> >> I think whether and when to deprecate Hadoop-1 is better to be moved to
> an
> >> official [DISCUSS] thread. What I disliked is the sentiment that I felt
> >> when we decide to deprecate something just because it's unfortunately
> >> broken for one reason or the other. Nevertheless, we should fix these
> bugs
> >> at this moment. We fixed such issues in previous release as well.
> >>
> >> I concur that we should have caught these breakages before they made
> their
> >> way into the code. Previously we discussed about running pre-commit
> tests
> >> alternatively or even randomly with either Hadoop-1 or Hadoop-2 profile.
> >> Now it seems critical to do so. Thus, I support Szehon's proposal to
> make
> >> this happen. This is orthogonal to the deprecation discussion.
> >>
> >> Thanks,
> >> Xuefu
> >>
> >> On Wed, Apr 29, 2015 at 11:30 AM, Szehon Ho <szehon@cloudera.com>
> wrote:
> >>
> >> > Are you
> >> > > personally willing to do the work to make sure that Hive works with
> >> > Hadoop
> >> > > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443,
> 10444)
> >> > > because
> >> > > trunk currently does not work with Hadoop 1.x and no one noticed.
> >> >
> >> > We setup a Hadoop-1 build sometime back and made sure a lot of
> breakage
> >> > were fixed, but had not hooked it up to HiveQA (precommit) due to
> >> concerns
> >> > of the time it would take to run these tests.  It was a balance
> between
> >> > getting faster HiveQA results for everyone and testing all the paths,
> >> even
> >> > today there are many paths we don't test.
> >> >
> >> > So if these JIRA's are fixed, would folks be amenable to alternate
> >> testing
> >> > hadoop-1 and hadoop-2 on HiveQA precommit runs and for devs to not
> commit
> >> > further breakages of Hadoop 1.x compat, for another release?  Yes I do
> >> > understand its a burden to maintain hadoop-1.
> >> >
> >> > if you use HDP, CDH or other hadoop distro then hadoop-1.x and
> hive-1.2
> >> > > combination does not exist.
> >> > >
> >> >
> >> > I think Apache is agnostic to specific vendors so should not discuss
> >> based
> >> > on that, but vendors still have this combination, I know of users that
> >> use
> >> > Hadoop-1 and latest Hive 1.1.
> >> >
> >> > Thanks,
> >> > Szehon
> >> >
> >> > On Wed, Apr 29, 2015 at 11:07 AM, Alexander Pivovarov <
> >> > apivovarov@gmail.com>
> >> > wrote:
> >> >
> >> > > HDP and CDH distro are based on hadoop-2.x since 2013-2014
> >> > >
> >> > > if you use HDP, CDH or other hadoop distro then hadoop-1.x and
> hive-1.2
> >> > > combination does not exist.
> >> > > hdp-1.3 has hive-0.11
> >> > > cdh-4.7 has hive-0.10
> >> > >
> >> > > The only way to install hive-1.2 on hadoop-1.x is to do it manually.
> >> > > But if you manually install hadoop components then most probably you
> >> will
> >> > > install hadoop-2.x
> >> > >
> >> > > So, we can say that hadoop-1.x and hive-1.2 combination should not
> >> exist
> >> > in
> >> > > real life scenario.
> >> > >
> >> > >
> >> > >
> >> > > On Wed, Apr 29, 2015 at 8:03 AM, Owen O'Malley <omalley@apache.org>
> >> > wrote:
> >> > >
> >> > > > On Tue, Apr 28, 2015 at 10:16 PM, Szehon Ho <szehon@cloudera.com>
> >> > wrote:
> >> > > >
> >> > > > > Yes, question is whether users want to run latest Hive version
> on
> >> > > Hadoop
> >> > > > > 1.x clusters.
> >> > > > >
> >> > > >
> >> > > > But you ignored the more important question that I asked. Are you
> >> > > > personally willing to do the work to make sure that Hive works
> with
> >> > > Hadoop
> >> > > > 1.x? There are five new jiras (HIVE-10430, 10431, 10442, 10443,
> >> 10444)
> >> > > > because
> >> > > > trunk currently does not work with Hadoop 1.x and no one noticed.
> >> > > >
> >> > > > Supporting Hadoop 1.x costs the Hive project a lot of work and
> >> provides
> >> > > > very little in return.
> >> > > >
> >> > > > .. Owen
> >> > > >
> >> > >
> >> >
> >>
>

--001a1134effa84bc550514e3248e--