geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Udo Kohlmeyer <ukohlme...@pivotal.io>
Subject Re: New client/server protocol - seeking feedback
Date Mon, 02 Oct 2017 20:19:25 GMT
+1 to what Jake said. Our approach is exactly what your preference is. The
adding of messages would new operations and variations on the operation.
PUT will be different to PUT_WITH_CALLBACK. Even in the backend the
processing of the messages will be handled by a different
operationsHandlers.

With the addition of the HANDSHAKE message GEODE-3705
<https://issues.apache.org/jira/browse/GEODE-3705> we hope to be able to
negotiate supported versions and functionality up-front, before processing
any operational messages.

In addition to that, major version changes will only be introduced with
non-backwards compatible changes to the message structure. How to multiplex
the different major version messages is still up for design and
implementation though.

--Udo

On Mon, Oct 2, 2017 at 12:37 PM, Michael William Dodge <mdodge@pivotal.io>
wrote:

> From my days using Win32 APIs, I think fixing Foo() with FooEx() is an
> anti-pattern. But that's not to say that "version 37 fixes the parameters
> to Foo() and in no other way changes anything" is any better. I see the
> version as useful for determining the structure of the protocol, not the
> specifics of a message per se.
>
> One of the disadvantages of using versions is that it can lead to
> spaghetti code such as cascading if statements to handle different versions
> of any given message. I worry that having the client and server negotiate
> which messages they are going to use would also be a significant addition
> of complexity.
>
> Sarge
>
> > On 2 Oct, 2017, at 11:14, Jacob Barrett <jbarrett@pivotal.io> wrote:
> >
> > A change to a message should just be a new message, no need to version
> it.
> > Clients and severs could negotiate the messages they support or attempt
> the
> > message they support and fallback to an alternative if the server rejects
> > it. Consider Put and PutEx (ignore the names):
> > Put ( Key, Value )
> > PutEx (Key, Value, SomethingElse )
> > The client could try PutEx but if rejected by older server and
> > SomethingElse is not important to its operation it could try Put instead.
> > Alternatively the server could be queried or a list of supported message
> > IDs in which it could return only PutEx and the older client could make a
> > decision easier as to whether or not it can talk to the server.
> >
> > Although one could argue these are district operation that should be
> > defined as independent messages anyway. Think clean OO design in your
> > message design. The message should have significantly change behavior
> > because of a single parameter otherwise it is really a new thing and
> should
> > be defined as a new message.
> >
> > The short answer is that version numbers make for a nightmare of
> > compatibility especially when interleaving releases and maintenance
> > releases. Look at our current protocol and the gaps we leave in the
> ordinal
> > numbering to avoid this issue. Let's not make that same mistake.
> >
> >
> >
> > As for interleaving requests and responses, this should be layered in the
> > protocol. The top layer should only deal with serial request/response.
> Let
> > a lower layer encapsulate the upper level in a multiplexed manor. The
> naive
> > layer could just open additional sockets to achieve interleaving, while
> an
> > advanced approach would create sub channels on the socket and forward to
> > the appropriate upper later session. All very easily achieved with Netty.
> > When the client connects it could negotiate if the server supports the
> > channel method, just like we negotiate SSL or authentication. The other
> > benefit to this approach is you don't have an unused field in your
> protocol
> > for clients that don't want to implement something so complex.
> >
> >
> > -Jake
> >
> >
> >
> >
> >
> > On Mon, Oct 2, 2017 at 10:22 AM Michael Stolz <mstolz@pivotal.io> wrote:
> >
> >> We should check that it is actually safe to add fields.
> >> If it isn't we're likely to have a lot of versioning to do.
> >>
> >> --
> >> Mike Stolz
> >> Principal Engineer, GemFire Product Lead
> >> Mobile: +1-631-835-4771 <(631)%20835-4771>
> >>
> >> On Mon, Sep 25, 2017 at 5:25 PM, Galen O'Sullivan <
> gosullivan@pivotal.io>
> >> wrote:
> >>
> >>> Replies inline.
> >>>
> >>> On Mon, Sep 25, 2017 at 12:13 PM, Udo Kohlmeyer <ukohlmeyer@pivotal.io
> >
> >>> wrote:
> >>>
> >>>> Replies inline
> >>>> On Mon, Sep 25, 2017 at 11:21 AM, Dan Smith <dsmith@pivotal.io>
> wrote:
> >>>>
> >>>>> This actually brings up another point I was going to ask about.
I
> >> don't
> >>>> see
> >>>>> any version information in the protocol. How will we handle adding
> >> new
> >>>>> features to this protocol? Do the clients and servers negotiate
which
> >>>>> version of the protocol to use somehow?
> >>>>>
> >>>>> I think there should be a plan in place for making changes to the
> >>>> protocol
> >>>>> and supporting old clients. Given that, we shouldn't add fields
that
> >>>> aren't
> >>>>> actually used into the existing version of the protocol. When we
> >>>> introduce
> >>>>> new features into the protocol, that's the point at which we should
> >> add
> >>>> the
> >>>>> fields to support that feature.
> >>>>>
> >>>>
> >>>> [UK] - Protobuf allows for the amending of messages. As soon as the
> >>>> protocol changes significantly the "magic" number will have to be
> >>>> incremented, indicating a new (non-backward compatible) version of the
> >>>> protocol. This of course has bigger problems, where Java does not
> allow
> >>> for
> >>>> multiple versions of the same class to be loaded, so a server could
> run
> >>>> only 1 version of Protobuf messages at a time.
> >>>>
> >>>
> >>> We have to be careful about how we extend protobuf messages, though.
> I'm
> >>> not sure exactly what's safe to do, but it should at least be safe to
> add
> >>> fields (assuming they don't change existing behavior -- we'll have to
> >> think
> >>> about this) and ignore old fields (which is how you would remove a
> >>> now-unused field). It's fairly simple to add new operations without any
> >>> interesting breakages - they'll fail with older servers and not be sent
> >>> with older clients. I think adding new operations is a pretty good way
> to
> >>> add features that don't require a real rework of the protocol. For
> those
> >>> that do, we can version the initial byte.
> >>>
> >>
>
>


-- 
Kindest Regards
-----------------------------
*Udo Kohlmeyer* | *Snr Solutions Architect* |*Pivotal*
*Mobile:* +61 409-279-160 | ukohlmeyer@pivotal.io
<http://www.gopivotal.com/>
www.pivotal.io

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message