hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sangjin Lee <sj...@apache.org>
Subject Re: Protocol Buffers version
Date Wed, 20 May 2015 03:40:32 GMT
When we moved to Hadoop 2.4, the associated protobuf upgrade (2.4.1 ->
2.5.0) proved to be one of the bigger problems. In our case, most of our
users were using protobuf 2.4.x or earlier.

We identified a couple of places where the backward compatibility was
broken, and patched for those issues. We've been running with that patched
version of protobuf 2.5.0 since. I can push out those changes to github or
something if others are interested FWIW.

Regards,
Sangjin

On Tue, May 19, 2015 at 9:59 AM, Colin P. McCabe <cmccabe@apache.org> wrote:

> I agree that the protobuf 2.4.1 -> 2.5.0 transition could have been
> handled a lot better by Google.  Specifically, since it was an
> API-breaking upgrade, it should have been a major version bump for the
> Java library version.  I also feel that removing the download links
> for the old versions of the native libraries was careless, and
> certainly burned some of our Hadoop users.
>
> However, I don't see any reason to believe that protobuf 2.6 will not
> be wire-compatible with earlier versions.  Google has actually been
> pretty good about preserving wire-compatibility... just not about API
> compatibility.  If we want to get a formal statement from the project,
> we can, but I would be pretty shocked if they decided to change the
> protocol in a backwards-incompatible way in a minor version release.
>
> I do think there are some potential issues for our users of bumping
> the library version in a minor Hadoop release.  Until we implement
> full dependency isolation for Hadoop, there may be some disruptions to
> end-users from changing Java dependency versions.  Similarly, users
> will need to install a new native protobuf library version as well.
> So I think we should bump the protobuf versions in Hadoop 3.0, but not
> in 2.x.
>
> cheers,
> Colin
>
> On Fri, May 15, 2015 at 4:55 AM, Alan Burlison <Alan.Burlison@oracle.com>
> wrote:
> > On 15/05/2015 09:44, Steve Loughran wrote:
> >
> >> Now: why do you want to use a later version of protobuf.jar? Is it
> >> because "it is there"? Or is there a tangible need?
> >
> >
> > No, it's because I'm looking at this from a platform perspective: We have
> > other consumers of ProtoBuf beside Hadoop and we'd obviously like to
> > minimise the versions of PB that we ship, and preferably just ship the
> > latest version. The fact that PB seems to often be incompatible across
> > releases is an issue as it makes upgrading and dropping older versions
> > problematic.
> >
> > --
> > Alan Burlison
> > --
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message