hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wang <andrew.w...@cloudera.com>
Subject Re: Can we update protobuf's version on trunk?
Date Tue, 28 Mar 2017 23:18:59 GMT
>
> > If unknown fields are dropped, then applications proxying tokens and
> other
> >> data between servers will effectively corrupt those messages, unless we
> >> make everything opaque bytes, which- absent the convenient, prenominate
> >> semantics managing the conversion- obviate the compatibility machinery
> that
> >> is the whole point of PB. Google is removing the features that justified
> >> choosing PB over its alternatives. Since we can't require that our
> >> applications compile (or link) against our updated schema, this creates
> a
> >> problem that PB was supposed to solve.
> >
> >
> > This is scary, and it potentially affects services outside of the Hadoop
> > codebase. This makes it difficult to assess the impact.
>
> Stack mentioned a compatibility mode that uses the proto2 semantics.
> If that carries unknown fields through intermediate handlers, then
> this objection goes away. -C


Did some more googling, found this:

https://groups.google.com/d/msg/protobuf/Z6pNo81FiEQ/fHkdcNtdAwAJ

Feng Xiao appears to be a Google engineer, and suggests workarounds like
packing the fields into a byte type. No mention of a PB2 compatibility
mode. Also here:

https://groups.google.com/d/msg/protobuf/bO2L6-_t91Q/-zIaJAR9AAAJ

Participants say that unknown fields were dropped for automatic JSON
encoding, since you can't losslessly convert to JSON without knowing the
type.

Unfortunately, it sounds like these are intrinsic differences with PB3.

Best,
Andrew

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message