hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: Protocol Buffers version
Date Wed, 20 May 2015 05:05:45 GMT
Thanks, Sangjin.  I'd be interested in taking a peek at a personal GitHub
repo or even just a patch file of those changes.  If there were
incompatibilities, then that doesn't bode well for an upgrade to 2.6.

--Chris Nauroth

On 5/19/15, 8:40 PM, "Sangjin Lee" <sjlee@apache.org> wrote:

>When we moved to Hadoop 2.4, the associated protobuf upgrade (2.4.1 ->
>2.5.0) proved to be one of the bigger problems. In our case, most of our
>users were using protobuf 2.4.x or earlier.
>We identified a couple of places where the backward compatibility was
>broken, and patched for those issues. We've been running with that patched
>version of protobuf 2.5.0 since. I can push out those changes to github or
>something if others are interested FWIW.
>On Tue, May 19, 2015 at 9:59 AM, Colin P. McCabe <cmccabe@apache.org>
>> I agree that the protobuf 2.4.1 -> 2.5.0 transition could have been
>> handled a lot better by Google.  Specifically, since it was an
>> API-breaking upgrade, it should have been a major version bump for the
>> Java library version.  I also feel that removing the download links
>> for the old versions of the native libraries was careless, and
>> certainly burned some of our Hadoop users.
>> However, I don't see any reason to believe that protobuf 2.6 will not
>> be wire-compatible with earlier versions.  Google has actually been
>> pretty good about preserving wire-compatibility... just not about API
>> compatibility.  If we want to get a formal statement from the project,
>> we can, but I would be pretty shocked if they decided to change the
>> protocol in a backwards-incompatible way in a minor version release.
>> I do think there are some potential issues for our users of bumping
>> the library version in a minor Hadoop release.  Until we implement
>> full dependency isolation for Hadoop, there may be some disruptions to
>> end-users from changing Java dependency versions.  Similarly, users
>> will need to install a new native protobuf library version as well.
>> So I think we should bump the protobuf versions in Hadoop 3.0, but not
>> in 2.x.
>> cheers,
>> Colin
>> On Fri, May 15, 2015 at 4:55 AM, Alan Burlison
>> wrote:
>> > On 15/05/2015 09:44, Steve Loughran wrote:
>> >
>> >> Now: why do you want to use a later version of protobuf.jar? Is it
>> >> because "it is there"? Or is there a tangible need?
>> >
>> >
>> > No, it's because I'm looking at this from a platform perspective: We
>> > other consumers of ProtoBuf beside Hadoop and we'd obviously like to
>> > minimise the versions of PB that we ship, and preferably just ship the
>> > latest version. The fact that PB seems to often be incompatible across
>> > releases is an issue as it makes upgrading and dropping older versions
>> > problematic.
>> >
>> > --
>> > Alan Burlison
>> > --

View raw message