geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ernest Burghardt <eburgha...@pivotal.io>
Subject Re: [gemfire-dev] New Client-Server Protocol Proposal
Date Mon, 08 May 2017 21:55:50 GMT
+1 William, an even better example - that kind of representation will make
it so much better/easier for geode users to implement against, regardless
of language.

On Mon, May 8, 2017 at 2:48 PM, William Markito Oliveira <
william.markito@gmail.com> wrote:

> +1
>
> I think I've shared this before, but Kafka also has good (tabular)
> representation for messages on their protocol.
>
> - https://kafka.apache.org/protocol#protocol_messages
> - https://kafka.apache.org/protocol#protocol_message_sets
>
> On Mon, May 8, 2017 at 4:44 PM, Ernest Burghardt <eburghardt@pivotal.io>
> wrote:
>
> > Hello Geodes!
> >
> > Good discussion on what/how the messages will be/handled once a
> connection
> > is established.
> >
> > +1 to a simple initial handshake to establish version/supported features
> > that client/server will be communicating.
> >
> > From what I've seen so far in the proposal it is missing a definition for
> > the "connection"/disconnect messages.
> > - expected to see it here:
> > https://cwiki.apache.org/confluence/display/GEODE/Generic+System+API
> >
> > From a protocol perspective, this is currently a pain point for the
> > geode-native library.
> >
> > As Jake mentioned previously, having messages that are class-like and
> have
> > a singular job helps client developers by having an explicit protocol to
> > follow.
> >
> >
> > The basic case a developer is going to exercise is to connect/disconnect.
> > How to do this should be straightforward from the start.
> >
> > Geode probably does not need a 7 Layer OSI stack, but it might make sense
> > to have a couple layers:
> >
> > 1 - transport  (network socket)
> > 2 - protocol   (version/features)
> > 3 - messaging (do cluster work)
> >
> > e.g.
> > client library opens a socket to the server (layer 1 - check)
> > client/server perform handshake and the connection is OPEN (layer 2 -
> > check)
> > pipe is open for business, client/server do work freely (layer 3 - check)
> >
> > When this is sorted out I think a couple simple sequence or activity
> > diagrams would be very helpful to the visual-spatial folks in the
> > community.
> >
> >
> > Best,
> > Ernie
> >
> > ps.  one consideration for message definition might be to use a more
> > tabular presentation of messages followed by any
> > definitions/cross-referencing... this is an example from a CDMA
> protocol I
> > have worked with in the past
> >
> > Location assignment message
> >
> > Field
> >
> > Length (bits)
> >
> > MessageID
> >
> > 8
> >
> > TransactionID
> >
> > 8
> >
> > LocationType
> >
> > 8
> >
> > LocationLength
> >
> > 8
> >
> > LocationValue
> >
> > 8 × LocationLength
> >
> >    1.
> >
> >    MessageID           The access network shall set this field to 0x05.
> >    2.
> >
> >    TransactionID      The access network shall increment this value for
> >    each new LocationAssignment message sent.
> >
> >           LocationType       The access network shall set this field to
> the
> > type of the location as specified in Table
> >
> >
> >
> >
> >
> > [image: page144image36968] [image: page144image37392] [image:
> > page144image37976] [image: page144image38560] [image:
> > page144image38984] [image:
> > page144image39408]
> >
> >
> >
> >
> >
> >
> >
> > On Mon, May 8, 2017 at 7:48 AM, Jacob Barrett <jbarrett@pivotal.io>
> wrote:
> >
> > > On Fri, May 5, 2017 at 2:09 PM Hitesh Khamesra <hiteshk25@yahoo.com>
> > > wrote:
> > >
> > > >
> > > > 0. In first phase we are not doing chunking/fragmentation. And even
> > this
> > > > will be option for client.(
> > > > https://cwiki.apache.org/confluence/display/GEODE/
> > Message+Structure+and+
> > > Definition#MessageStructureandDefinition-Protocolnegotiation
> > > > )
> > > >
> > >
> > > I highly suggest initial handshake be more relaxed than specific
> "version
> > > number" or flags. Consider sending objects that indicate support for
> > > features or even a list of feature IDs. At connect server can send list
> > of
> > > feature IDs to the client. The client can respond with a set of feature
> > IDs
> > > it supports as well as any metadata associated with them, say default
> set
> > > of supported encodings.
> > >
> > >
> > > > 1. Are you refereeing websocket/spdy? But I think we are talking
> almost
> > > > same thing, may be push isPartialMessage flag with chunk
> > length(Anthony's
> > > > example below) ?
> > > >
> > >
> > > I am not sure what you mean here but if you are talking about layering
> a
> > > channel protocol handler then I guess yes. The point is that each of
> > these
> > > behaviors should be encapsulated in specific layers and not intermixed
> > with
> > > the message.
> > >
> > >
> > > > 2. That's the part of the problem. Even if you need to serialize the
> > > > "String", you need to write length first and then need to write
> > > serialized
> > > > utf bytes. We can implement chunked input stream and can de-serialize
> > the
> > > > object as it is coming (DataSerializable.fromData(ChunkedStream)).
> > > >
> > >
> > > Right, and in this case the length is never the length of the string,
> it
> > is
> > > the length of the byte encoding of the string. This is not known until
> > the
> > > encoding is complete. So by chunking we can write the length of smaller
> > > buffers (from buffer pools) as the length of that sequence of bytes,
> the
> > > last chunk terminated with length 0. Each of those chunks can be based
> > to a
> > > UTF-8 to UTF-16 transcoder to create the String.
> > >
> > > -Jake
> > >
> >
>
>
>
> --
> ~/William
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message