Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of todd@cloudera.com designates
 209.85.160.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAKfebOPb8miLsSQda0=rf4jP70+HdNxfnv9hShc8TBAjCDydTw@mail.gmail.com>
References: 
 <CAPz5eN6+=mZ36er-AR3Hkv5UP0DkxoU2x97a_XnU+NGkGuo+pw@mail.gmail.com>
 <EDF15D04-D07E-4C03-8FC4-10785965CCF4@hortonworks.com>
 <CAKfebOMYaSnKjFo8yAHFmoyrSFURvb0W3TxVoniVf-Je1ARhZQ@mail.gmail.com>
 <1344385395.72355.YahooMailNeo@web121705.mail.ne1.yahoo.com>
 <CAKfebOPb8miLsSQda0=rf4jP70+HdNxfnv9hShc8TBAjCDydTw@mail.gmail.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Wed, 8 Aug 2012 12:46:42 -0700
Message-ID: 
 <CADY20s4mRC4-NzQeKK0XavmzmBQhExbHbynKy=f46ZwxXZCvow@mail.gmail.com>
Subject: Re: Use of required keyword in protobuf messages
To: dev@hbase.apache.org
Cc: lars hofhansl <lhofhansl@yahoo.com>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Yea, my recollection is that we decided to leave rolling upgrade
across major versions out of scope for now, but like Greg said, try to
do it between any two major versions where it looks "easy enough".

-Todd

On Wed, Aug 8, 2012 at 12:39 PM, Gregory Chanan <gchanan@cloudera.com> wrot=
e:
> Good question, Lars.
>
> I'm going off of the proposal (
> http://wiki.apache.org/hadoop/Hbase/HBaseWireCompatibility
> ) and the slides at the meetup (
> http://files.meetup.com/1350427/wire-compat%20%281%29.pptx).  If I missed
> anything, I apologize.
>
> Those list the minimum requirements to support the use cases and goals on
> slides 4-6.  So, as written, there is no *guarantee* of support for rolli=
ng
> upgrade between major versions.
>
> Of course, there is nothing stopping us from adding that guarantee.  Or
> from just guaranteeing it for specific versions (e.g. 0.96 to 0.98 in the
> same manner as 0.92 to 0.94).  It should be much easier to achieve the
> latter than in the past, once everything has been protobufed.  Perhaps we
> should discuss in another thread?
>
> So I'll withdraw my point that perhaps we could treat server-server and
> client-server communication differently, but still submit the point that =
we
> don't need to keep required fields around forever.
>
> Greg
>
> On Tue, Aug 7, 2012 at 5:23 PM, lars hofhansl <lhofhansl@yahoo.com> wrote=
:
>
>> Huh?
>>
>> I thought we're allowing for a rolling upgrade between major versions.
>> If server/server is only wire compatible across a minor version that won=
't
>> be possible.
>>
>> (this might have been mentioned in the initial discussion we all had abo=
ut
>> this a while back and I might have just forgotten)
>>
>>
>> -- Lars
>>
>>
>> ----- Original Message -----
>> From: Gregory Chanan <gchanan@cloudera.com>
>> To: dev@hbase.apache.org
>> Cc:
>> Sent: Tuesday, August 7, 2012 1:33 PM
>> Subject: Re: Use of required keyword in protobuf messages
>>
>> I agree with most of the comments here, particularly that at some point =
we
>> should go through and review all the "required" fields.
>>
>> Another thing to keep in mind is that we have only guaranteed the follow=
ing
>> are compatible:
>> - Client/Server across 1 major version
>> - Server/Server different minor versions
>>
>> So we don't need to keep required fields for eons, necessarily.  And it =
may
>> make sense to be looser about allowing "required" for server/server
>> communication than for client/server.
>>
>> Greg
>>
>> On Tue, Aug 7, 2012 at 11:39 AM, Devaraj Das <ddas@hortonworks.com> wrot=
e:
>>
>> > I too tend to make fields optional unless I am really convinced that t=
he
>> > field would live for eons. I agree with Google's philosophy in that
>> regard.
>> >
>> > On Aug 6, 2012, at 3:39 PM, Chris Trezzo wrote:
>> >
>> > > Hi All,
>> > >
>> > > I was looking through the .proto files and noticed there are a lot o=
f
>> > > fields that are marked as required. I am by no means a protobuf expe=
rt,
>> > but
>> > > I was wondering what advantage do we actually get in making fields
>> > required?
>> > >
>> > > I understand that if we don't use the required keyword we would have=
 to
>> > > implement custom application logic, but the flexibility we gain from
>> > having
>> > > all the fields optional seems to outweigh that work. In addition, we
>> will
>> > > already have to add logic to HBase to handle version compatibility, =
so
>> it
>> > > seems natural to implement the required logic as part of that layer.
>> This
>> > > would allow us to change or delete any message field and maintain wi=
re
>> > > compatibility.
>> > >
>> > > Quote from the protobuf language guide (
>> > > https://developers.google.com/protocol-buffers/docs/proto):
>> > >
>> > > "*Required Is Forever* You should be very careful about marking fiel=
ds
>> as
>> > > required. If at some point you wish to stop writing or sending a
>> required
>> > > field, it will be problematic to change the field to an optional fie=
ld
>> =96
>> > > old readers will consider messages without this field to be incomple=
te
>> > and
>> > > may reject or drop them unintentionally. You should consider writing
>> > > application-specific custom validation routines for your buffers
>> instead.
>> > > Some engineers at Google have come to the conclusion that using
>> > > requireddoes more harm than good; they prefer to use only
>> > > optional and repeated. However, this view is not universal."
>> > >
>> > > Thoughts?
>> > >
>> > > Thanks,
>> > > Chris Trezzo
>> >
>> >
>>


--=20
Todd Lipcon
Software Engineer, Cloudera