reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Weimer <mar...@weimo.de>
Subject Re: Protobuf version upgrade?
Date Thu, 08 Sep 2016 21:48:50 GMT
Yeah, this is a bit of a mess. We initially pushed for Avro because
(a) the main version is from Apache and (b) there is a .NET version
from Microsoft available under the ASL. Protobuf did not exist for
.NET at the time. It now does, and the Microsoft implementation seems
abandoned and won't make it to CoreCLR.

Hence, we are flip-flopping on the question of what the best
serializer for REEF's use case is. Right now, protobuf seems like the
only one available for Java and .NET (including CoreCLR).

Markus

On Thu, Sep 8, 2016 at 2:45 PM, Byung-Gon Chun <bgchun@gmail.com> wrote:
> Douglas,
>
> Do we want to completely remove the Avro dependency?
> At some point, we decided to take the Avro dependency by removing protobuf.
>
> -Gon
>
> On Thu, Sep 8, 2016 at 2:30 PM, Douglas Service <dsopsrc@gmail.com> wrote:
>
>> This appears to be best way to remove the Avro dependency as we port to
>> .NET Core.
>>
>> On Wed, Sep 7, 2016 at 8:29 AM, Markus Weimer <markus@weimo.de> wrote:
>>
>> > On 2016-09-02 3:34 PM, Douglas Service wrote:
>> >
>> >> http://stackoverflow.com/questions/34487996/how-can-i-use-
>> >> proto3-with-hadoop-spark
>> >>
>> >
>> > The top answer there is to use Shading in the resulting JAR. That works,
>> > but comes with all the downsides of shading. I'm OK with this, as long as
>> > we document this really well on our website such that our users know if /
>> > when to shade protobuf in their own builds.
>> >
>> > What do others think?
>> >
>> > Markus
>> >
>>
>
>
>
> --
> Byung-Gon Chun

Mime
View raw message