Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of tucu@cloudera.com designates
 209.85.216.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CANJr2=HLcUcsxaC+xPCAeg-DWCfiwg8mU-=ZSgz82Cc0sNxsVQ@mail.gmail.com>
References: 
 <CAJs-t7Nw6TpX7Nd1m3mqCz6PO9kWY6Uh_h-ppwA1WEoXDFk=eQ@mail.gmail.com>
 <CAAD07O+eDKz1kXOab5R2TZK-b1=iu6i8ZCZh3-us+N0U6wW6UQ@mail.gmail.com>
 <1375975131.91655.YahooMailNeo@web126005.mail.ne1.yahoo.com>
 <CALvzVW0qimqy_vrkHR=wndpP2mNTPOzF=jQjrVy-2f6USYGuRQ@mail.gmail.com>
 <CAJs-t7PjTD-3PbOVoDHmWNPxXy_Nn17+y5_d07tp5feCchcaFg@mail.gmail.com>
 <CAJs-t7OrzG8PTtaw1XxMvc8ep1CZbByUgcHhoDroKQHoSfu8+A@mail.gmail.com>
 <CANJr2=GLz-3WO6gtnVzcxwbmdUX_czYGBHEuch3B0zm0diKZRQ@mail.gmail.com>
 <CANJr2=HoUcnjVYaX38+OiELtXf2o4di9OJmccV75zVeGqUprtA@mail.gmail.com>
 <3038D364-C4EC-4047-B96C-48991DB1654E@cloudera.com>
 <CANJr2=HLcUcsxaC+xPCAeg-DWCfiwg8mU-=ZSgz82Cc0sNxsVQ@mail.gmail.com>
From: Alejandro Abdelnur <tucu@cloudera.com>
Date: Mon, 12 Aug 2013 11:37:21 -0700
Message-ID: 
 <CAJs-t7MY5bKxkpuGeTSrLkBiZEc1jLh6+5bRDCQzajo7tAghWA@mail.gmail.com>
Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845
To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Cc: "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>,
	"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>,
 Kihwal Lee <kihwal@yahoo-inc.com>,
	"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=047d7bdc7f060bc45704e3c46c87

--047d7bdc7f060bc45704e3c46c87
Content-Type: text/plain; charset=ISO-8859-1

Giri,

first of all, thanks for installing protoc 2.5.0.

I didn't know we were installing them as the only version and not driven by
env/path settings.

Now we have a bit of a problem, precommit builds are broken because of
mismatch of protoc (2.5.0) and protobuf JAR( 2.4.1).

We have to options:

1* commit HADOOP-9845 that will bring protobuf to 2.5.0 and iron out any
follow up issues.
2* reinstall protoc 2.4.1 in the jenkins machines and have 2.4.1 and 2.5.0
coexisting

My take would be to commit HADOOP-9845 in trunk, iron out any issues an
then merge it to the other branches.

We need to sort this out quickly as precommits are not working.

I'll wait till 3PM today  for objections to option #1, if none I'll commit
it to trunk.

Thanks.

Alejandro


On Mon, Aug 12, 2013 at 11:30 AM, Giridharan Kesavan <
gkesavan@hortonworks.com> wrote:

> Like I said protoc is upgraded from 2.4 to 2.5. 2.5 is in the default path.
> If we still need 2.4 I may have to install it. Let me know
>
> -Giri
>
>
> On Sat, Aug 10, 2013 at 7:01 AM, Alejandro Abdelnur <tucu@cloudera.com
> >wrote:
>
> > thanks giri, how do we set 2.4 or 2.5., what is the path to both so we
> can
> > use and env to set it in the jobs?
> >
> > thx
> >
> > Alejandro
> > (phone typing)
> >
> > On Aug 9, 2013, at 23:10, Giridharan Kesavan <gkesavan@hortonworks.com>
> > wrote:
> >
> > > build slaves hadoop1-hadoop9 now has libprotoc 2.5.0
> > >
> > >
> > >
> > > -Giri
> > >
> > >
> > > On Fri, Aug 9, 2013 at 10:56 PM, Giridharan Kesavan <
> > > gkesavan@hortonworks.com> wrote:
> > >
> > >> Alejandro,
> > >>
> > >> I'm upgrading protobuf on slaves hadoop1-hadoop9.
> > >>
> > >> -Giri
> > >>
> > >>
> > >> On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur <tucu@cloudera.com
> > >wrote:
> > >>
> > >>> pinging again, I need help from somebody with sudo access to the
> hadoop
> > >>> jenkins boxes to do this or to get sudo access for a couple of hours
> to
> > >>> set
> > >>> up myself.
> > >>>
> > >>> Please!!!
> > >>>
> > >>> thx
> > >>>
> > >>>
> > >>> On Thu, Aug 8, 2013 at 2:29 PM, Alejandro Abdelnur <
> tucu@cloudera.com
> > >>>> wrote:
> > >>>
> > >>>> To move forward with this we need protoc 2.5.0 in the apache hadoop
> > >>>> jenkins boxes.
> > >>>>
> > >>>> Who can help with this? I assume somebody at Y!, right?
> > >>>>
> > >>>> Thx
> > >>>>
> > >>>>
> > >>>> On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark <eclark@apache.org>
> > >>> wrote:
> > >>>>
> > >>>>> In HBase land we've pretty well discovered that we'll need to have
> > the
> > >>>>> same version of protobuf that the HDFS/Yarn/MR servers are running.
> > >>>>> That is to say there are issues with ever having 2.4.x and 2.5.x on
> > >>>>> the same class path.
> > >>>>>
> > >>>>> Upgrading to 2.5.x would be great, as it brings some new classes we
> > >>>>> could use.  With that said HBase is getting pretty close to a
> rather
> > >>>>> large release (0.96.0 aka The Singularity) so getting this in
> sooner
> > >>>>> rather than later would be great.  If we could get this into 2.1.0
> it
> > >>>>> would be great as that would allow us to have a pretty easy story
> to
> > >>>>> users with regards to protobuf version.
> > >>>>>
> > >>>>> On Thu, Aug 8, 2013 at 8:18 AM, Kihwal Lee <kihwal@yahoo-inc.com>
> > >>> wrote:
> > >>>>>> Sorry to hijack the thread but, I also wanted to mention Avro. See
> > >>>>> HADOOP-9672.
> > >>>>>> The version we are using has memory leak and inefficiency issues.
> > >>> We've
> > >>>>> seen users running into it.
> > >>>>>>
> > >>>>>> Kihwal
> > >>>>>>
> > >>>>>>
> > >>>>>> ________________________________
> > >>>>>> From: Tsuyoshi OZAWA <ozawa.tsuyoshi@gmail.com>
> > >>>>>> To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
> > >>>>>> Cc: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>; "
> > >>>>> yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>; "
> > >>>>> mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>
> > >>>>>> Sent: Thursday, August 8, 2013 1:59 AM
> > >>>>>> Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release,
> > >>>>> HADOOP-9845
> > >>>>>>
> > >>>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> About Hadoop, Harsh is dealing with this problem in HADOOP-9346.
> > >>>>>> For more detail, please see the JIRA ticket:
> > >>>>>> https://issues.apache.org/jira/browse/HADOOP-9346
> > >>>>>>
> > >>>>>> - Tsuyoshi
> > >>>>>>
> > >>>>>> On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur <
> > >>> tucu@cloudera.com>
> > >>>>> wrote:
> > >>>>>>> I' like to upgrade to protobuf 2.5.0 for the 2.1.0 release.
> > >>>>>>>
> > >>>>>>> As mentioned in HADOOP-9845, Protobuf 2.5 has significant
> benefits
> > >>> to
> > >>>>>>> justify the upgrade.
> > >>>>>>>
> > >>>>>>> Doing the upgrade now, with the first beta, will make things
> easier
> > >>> for
> > >>>>>>> downstream projects (like HBase) using protobuf and adopting
> Hadoop
> > >>> 2.
> > >>>>> If
> > >>>>>>> we do the upgrade later, downstream projects will have to
> support 2
> > >>>>>>> different versions and they my get in nasty waters due to
> classpath
> > >>>>> issues.
> > >>>>>>>
> > >>>>>>> I've locally tested the patch in a pseudo deployment of
> 2.1.0-beta
> > >>>>> branch
> > >>>>>>> and it works fine (something is broken in trunk in the RPC layer
> > >>>>> YARN-885).
> > >>>>>>>
> > >>>>>>> Now, to do this it will require a few things:
> > >>>>>>>
> > >>>>>>> * Make sure protobuf 2.5.0 is available in the jenkins box
> > >>>>>>> * A follow up email to dev@ aliases indicating developers should
> > >>>>> install
> > >>>>>>> locally protobuf 2.5.0
> > >>>>>>>
> > >>>>>>> Thanks.
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Alejandro
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Alejandro
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Alejandro
> > >>
> > >>
> >
>


-- 
Alejandro

--047d7bdc7f060bc45704e3c46c87--