hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Giridharan Kesavan <gkesa...@hortonworks.com>
Subject Re: Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845
Date Mon, 12 Aug 2013 18:47:25 GMT
I can take care of re-installing 2.4 and installing 2.5 in a different
location. This would fix 2.0 branch builds as well.
Thoughts?

-Giri


On Mon, Aug 12, 2013 at 11:37 AM, Alejandro Abdelnur <tucu@cloudera.com>wrote:

> Giri,
>
> first of all, thanks for installing protoc 2.5.0.
>
> I didn't know we were installing them as the only version and not driven by
> env/path settings.
>
> Now we have a bit of a problem, precommit builds are broken because of
> mismatch of protoc (2.5.0) and protobuf JAR( 2.4.1).
>
> We have to options:
>
> 1* commit HADOOP-9845 that will bring protobuf to 2.5.0 and iron out any
> follow up issues.
> 2* reinstall protoc 2.4.1 in the jenkins machines and have 2.4.1 and 2.5.0
> coexisting
>
> My take would be to commit HADOOP-9845 in trunk, iron out any issues an
> then merge it to the other branches.
>
> We need to sort this out quickly as precommits are not working.
>
> I'll wait till 3PM today  for objections to option #1, if none I'll commit
> it to trunk.
>
> Thanks.
>
> Alejandro
>
>
>
> On Mon, Aug 12, 2013 at 11:30 AM, Giridharan Kesavan <
> gkesavan@hortonworks.com> wrote:
>
> > Like I said protoc is upgraded from 2.4 to 2.5. 2.5 is in the default
> path.
> > If we still need 2.4 I may have to install it. Let me know
> >
> > -Giri
> >
> >
> > On Sat, Aug 10, 2013 at 7:01 AM, Alejandro Abdelnur <tucu@cloudera.com
> > >wrote:
> >
> > > thanks giri, how do we set 2.4 or 2.5., what is the path to both so we
> > can
> > > use and env to set it in the jobs?
> > >
> > > thx
> > >
> > > Alejandro
> > > (phone typing)
> > >
> > > On Aug 9, 2013, at 23:10, Giridharan Kesavan <gkesavan@hortonworks.com
> >
> > > wrote:
> > >
> > > > build slaves hadoop1-hadoop9 now has libprotoc 2.5.0
> > > >
> > > >
> > > >
> > > > -Giri
> > > >
> > > >
> > > > On Fri, Aug 9, 2013 at 10:56 PM, Giridharan Kesavan <
> > > > gkesavan@hortonworks.com> wrote:
> > > >
> > > >> Alejandro,
> > > >>
> > > >> I'm upgrading protobuf on slaves hadoop1-hadoop9.
> > > >>
> > > >> -Giri
> > > >>
> > > >>
> > > >> On Fri, Aug 9, 2013 at 1:15 PM, Alejandro Abdelnur <
> tucu@cloudera.com
> > > >wrote:
> > > >>
> > > >>> pinging again, I need help from somebody with sudo access to the
> > hadoop
> > > >>> jenkins boxes to do this or to get sudo access for a couple of
> hours
> > to
> > > >>> set
> > > >>> up myself.
> > > >>>
> > > >>> Please!!!
> > > >>>
> > > >>> thx
> > > >>>
> > > >>>
> > > >>> On Thu, Aug 8, 2013 at 2:29 PM, Alejandro Abdelnur <
> > tucu@cloudera.com
> > > >>>> wrote:
> > > >>>
> > > >>>> To move forward with this we need protoc 2.5.0 in the apache
> hadoop
> > > >>>> jenkins boxes.
> > > >>>>
> > > >>>> Who can help with this? I assume somebody at Y!, right?
> > > >>>>
> > > >>>> Thx
> > > >>>>
> > > >>>>
> > > >>>> On Thu, Aug 8, 2013 at 2:24 PM, Elliott Clark <eclark@apache.org>
> > > >>> wrote:
> > > >>>>
> > > >>>>> In HBase land we've pretty well discovered that we'll
need to
> have
> > > the
> > > >>>>> same version of protobuf that the HDFS/Yarn/MR servers
are
> running.
> > > >>>>> That is to say there are issues with ever having 2.4.x
and 2.5.x
> on
> > > >>>>> the same class path.
> > > >>>>>
> > > >>>>> Upgrading to 2.5.x would be great, as it brings some new
classes
> we
> > > >>>>> could use.  With that said HBase is getting pretty close
to a
> > rather
> > > >>>>> large release (0.96.0 aka The Singularity) so getting
this in
> > sooner
> > > >>>>> rather than later would be great.  If we could get this
into
> 2.1.0
> > it
> > > >>>>> would be great as that would allow us to have a pretty
easy story
> > to
> > > >>>>> users with regards to protobuf version.
> > > >>>>>
> > > >>>>> On Thu, Aug 8, 2013 at 8:18 AM, Kihwal Lee <kihwal@yahoo-inc.com
> >
> > > >>> wrote:
> > > >>>>>> Sorry to hijack the thread but, I also wanted to mention
Avro.
> See
> > > >>>>> HADOOP-9672.
> > > >>>>>> The version we are using has memory leak and inefficiency
> issues.
> > > >>> We've
> > > >>>>> seen users running into it.
> > > >>>>>>
> > > >>>>>> Kihwal
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> ________________________________
> > > >>>>>> From: Tsuyoshi OZAWA <ozawa.tsuyoshi@gmail.com>
> > > >>>>>> To: "common-dev@hadoop.apache.org" <
> common-dev@hadoop.apache.org>
> > > >>>>>> Cc: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>;
> "
> > > >>>>> yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>;
"
> > > >>>>> mapreduce-dev@hadoop.apache.org" <
> mapreduce-dev@hadoop.apache.org>
> > > >>>>>> Sent: Thursday, August 8, 2013 1:59 AM
> > > >>>>>> Subject: Re: Upgrade to protobuf 2.5.0 for the 2.1.0
release,
> > > >>>>> HADOOP-9845
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Hi,
> > > >>>>>>
> > > >>>>>> About Hadoop, Harsh is dealing with this problem in
HADOOP-9346.
> > > >>>>>> For more detail, please see the JIRA ticket:
> > > >>>>>> https://issues.apache.org/jira/browse/HADOOP-9346
> > > >>>>>>
> > > >>>>>> - Tsuyoshi
> > > >>>>>>
> > > >>>>>> On Thu, Aug 8, 2013 at 1:49 AM, Alejandro Abdelnur
<
> > > >>> tucu@cloudera.com>
> > > >>>>> wrote:
> > > >>>>>>> I' like to upgrade to protobuf 2.5.0 for the 2.1.0
release.
> > > >>>>>>>
> > > >>>>>>> As mentioned in HADOOP-9845, Protobuf 2.5 has
significant
> > benefits
> > > >>> to
> > > >>>>>>> justify the upgrade.
> > > >>>>>>>
> > > >>>>>>> Doing the upgrade now, with the first beta, will
make things
> > easier
> > > >>> for
> > > >>>>>>> downstream projects (like HBase) using protobuf
and adopting
> > Hadoop
> > > >>> 2.
> > > >>>>> If
> > > >>>>>>> we do the upgrade later, downstream projects will
have to
> > support 2
> > > >>>>>>> different versions and they my get in nasty waters
due to
> > classpath
> > > >>>>> issues.
> > > >>>>>>>
> > > >>>>>>> I've locally tested the patch in a pseudo deployment
of
> > 2.1.0-beta
> > > >>>>> branch
> > > >>>>>>> and it works fine (something is broken in trunk
in the RPC
> layer
> > > >>>>> YARN-885).
> > > >>>>>>>
> > > >>>>>>> Now, to do this it will require a few things:
> > > >>>>>>>
> > > >>>>>>> * Make sure protobuf 2.5.0 is available in the
jenkins box
> > > >>>>>>> * A follow up email to dev@ aliases indicating
developers
> should
> > > >>>>> install
> > > >>>>>>> locally protobuf 2.5.0
> > > >>>>>>>
> > > >>>>>>> Thanks.
> > > >>>>>>>
> > > >>>>>>> --
> > > >>>>>>> Alejandro
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> Alejandro
> > > >>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Alejandro
> > > >>
> > > >>
> > >
> >
>
>
>
> --
> Alejandro
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message