Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CAABS=BNapp9Oi46RQLgagyGzSdmCDEaESnfJe5AF_buxLH4TsA@mail.gmail.com>
References: <CAK2_iMOi3=RVszTTeS_=-VG1tpvwB7pozD1AGOSrtrqD+-kybg@mail.gmail.com>
 <CAABS=BNtwQi=ZQF76VZT3wLhkjxeKjaN1z6gLByBS5xT7cvCGg@mail.gmail.com>
 <CAMbfdjYHY_61Xx4pi8rGaswSx-rxgDCe5btcMSU6sJDRWe2nVw@mail.gmail.com>
 <CAK2_iMPYzcuNsM4tno5tv5xVzvX4PBZZ_e+2NF+cVH_0ttDv6w@mail.gmail.com>
 <CAMbfdjaXE3L9cpekdvBm3bJNJ8T9+epZWPpxce8=XwxpUThDKA@mail.gmail.com>
 <CAMbfdja7EpqXzm467wXjwZvLOEq5OwPRz2dM48dy3Pk3YrrsdQ@mail.gmail.com>
 <CAK2_iMMHw82rdWMC0=YcDKBmj2Jc5Rns63-ayGGGxfKoChVu+Q@mail.gmail.com>
 <CAABS=BPY9=nbPQV0cwbhz5SkKeL3WZFtnMtr=uS_Gz6EXgreSA@mail.gmail.com>
 <CAK2_iMOCQ3gy_Y3vqwf+Y5tVK1a=jSp2PojXozQkiF5XkeVy_Q@mail.gmail.com> <CAABS=BNapp9Oi46RQLgagyGzSdmCDEaESnfJe5AF_buxLH4TsA@mail.gmail.com>
From: Pavel Tupitsyn <ptupitsyn@apache.org>
Date: Thu, 30 Nov 2017 10:42:31 +0300
Message-ID: <CAK2_iMMZCvVPq45QET9ANZCRHGUTUpD=ErHBcYet_ezSAUt5Cw@mail.gmail.com>
Subject: Re: Thin Client Protocol documentation
To: Prachi Garg <pgarg@gridgain.com>
Cc: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary="089e0825888cd2dbf0055f2e660d"
archived-at: Thu, 30 Nov 2017 07:43:07 -0000

--089e0825888cd2dbf0055f2e660d
Content-Type: text/plain; charset="UTF-8"

Prachi, see "Response (failure)" table.
Failed response includes server protocol version and an error message.

On Thu, Nov 30, 2017 at 4:55 AM, Prachi Garg <pgarg@gridgain.com> wrote:

> Pavel,
>
> If the connection handshake fails, what should be the message length in
> the response. When I try to fail the handshake, I get 32 as the message
> length. Is this expected?
>
> -Prachi
>
> On Wed, Nov 29, 2017 at 8:20 AM, Pavel Tupitsyn <ptupitsyn@gridgain.com>
> wrote:
>
>> Hi Prachi,
>>
>> Flags parameter in all cache operations is reserved for things like
>> withSkipStore, withExpiryPolicy, withKeepBinary, withNoRetries,
>> withPartitionRecover.
>> See methods in IgniteCache interface.
>>
>> Thanks,
>> Pavel
>>
>> On Wed, Nov 29, 2017 at 7:10 PM, Prachi Garg <pgarg@gridgain.com> wrote:
>>
>>> Hi Pavel,
>>>
>>> In the operation request, what does the 'flags' parameter mean?
>>>
>>>
>>> Thanks,
>>> -Prachi
>>>
>>> On Wed, Nov 29, 2017 at 5:27 AM, Pavel Tupitsyn <ptupitsyn@apache.org>
>>> wrote:
>>>
>>>> Sergey, good point, done.
>>>>
>>>> On Wed, Nov 29, 2017 at 2:30 PM, Sergey Kozlov <skozlov@gridgain.com>
>>>> wrote:
>>>>
>>>> > Pavel
>>>> >
>>>> > Could you update the page by following:
>>>> >
>>>> >  - String, date, UUID arrays allow to put NULL. Due to that every
>>>> item in
>>>> > the array written as type code byte (default item type or null type
>>>> code) +
>>>> > type data. It should be detailed explained (looks like that the table
>>>> > should have an addtional column called for instance "nullable")
>>>> >
>>>> > -  UUID type takes 16 bytes length
>>>> >
>>>> > thanks
>>>> >
>>>> >
>>>> > On Mon, Nov 27, 2017 at 4:37 PM, Sergey Kozlov <skozlov@gridgain.com>
>>>> > wrote:
>>>> >
>>>> > > Pavel
>>>> > >
>>>> > > Thanks for explanations!
>>>> > >
>>>> > > On Mon, Nov 27, 2017 at 2:46 PM, Pavel Tupitsyn <
>>>> ptupitsyn@apache.org>
>>>> > > wrote:
>>>> > >
>>>> > >> Sergey,
>>>> > >>
>>>> > >> 1. Code table size does not affect anything, as I understand, so
>>>> there
>>>> > is
>>>> > >> no reason to introduce an extra byte.
>>>> > >> 2. We have object arrays (code 23), I forgot to mention them,
>>>> fixed.
>>>> > >> 3. Also forgot, see code 25 in the updated document.
>>>> > >>
>>>> > >> Also note that operation codes have been updated (grouped by
>>>> purpose) as
>>>> > >> part of https://issues.apache.org/jira/browse/IGNITE-6989.
>>>> > >>
>>>> > >> Thanks,
>>>> > >> Pavel
>>>> > >>
>>>> > >> On Sun, Nov 26, 2017 at 9:54 PM, Sergey Kozlov <
>>>> skozlov@gridgain.com>
>>>> > >> wrote:
>>>> > >>
>>>> > >> > Pavel
>>>> > >> >
>>>> > >> > Thanks for the document and your efforts for new protocol. It was
>>>> > really
>>>> > >> > helpful for playing around the python thin client design.
>>>> > >> >
>>>> > >> > Could you explain some things that were still not clear for
>>>> binary
>>>> > >> object
>>>> > >> > format:
>>>> > >> >
>>>> > >> > 1. What a reason to introduce separated type codes for arrays?
>>>> Why
>>>> > just
>>>> > >> we
>>>> > >> > can't use the following?
>>>> > >> > *<1 byte universal array code>*
>>>> > >> > *<1 byte primitive code>*
>>>> > >> > *<4 bytes length>*
>>>> > >> > *<N bytes data>*
>>>> > >> >
>>>> > >> > We get 1 byte overhead but save 10 bytes in the code table. For
>>>> arrays
>>>> > >> the
>>>> > >> > overhead is really insignificant:10 longs array takes now
>>>> 1+4+4*10=45
>>>> > >> bytes
>>>> > >> > vs 1+1+4+4*10=46 bytes for the approach
>>>> > >> > Moreover for that appoach a new primitive code will be available
>>>> for
>>>> > >> using
>>>> > >> > for array immediately.
>>>> > >> >
>>>> > >> > 2. Why the arrays force to use a selected type? For python
>>>> there's no
>>>> > >> > limitations to use different types across one array (list).
>>>> Would be
>>>> > >> good
>>>> > >> > to introduce a new type that will allow that. It could be look
>>>> like
>>>> > >> > following
>>>> > >> > *<1 byte universal array code>*
>>>> > >> > *<1 byte no common type code*> <-- this says that every item must
>>>> > >> provide
>>>> > >> > its date type code like it does regular primitive data
>>>> > >> > *<4 bytes length>*
>>>> > >> > *<1 byte item 0 type code>*     <-- item provides its code
>>>> > >> > *<N byte item 0 data>*      <-- item provides its data
>>>> > >> > *<1 byte item 1 type code>*
>>>> > >> > *<N byte item 1 data>*
>>>> > >> > etc
>>>> > >> >
>>>> > >> > Also that allow to put nested arrays without changes in type code
>>>> > table!
>>>> > >> > For instance if we want to store 9 longs and 1 boolean it will
>>>> take
>>>> > >> > now 1+1+4+(1+9)*4+(1+1)=48
>>>> > >> > bytes (vs 45 bytes to store as 10 longs as usual).
>>>> > >> >
>>>> > >> > 3. Ther's only one way to store a dictionary (key-value)
>>>> structure as
>>>> > >> value
>>>> > >> > in the cache via Complex Object. But it looks like
>>>> overcomplicated. I
>>>> > >> > suppose to introduce a code for that
>>>> > >> > *<1 byte key-value dictionary code>*
>>>> > >> >
>>>> > >> > *<4 bytes length>*
>>>> > >> > *<1 byte key 1 **name **type code>*
>>>> > >> > *<N byte key 1 name data>*
>>>> > >> > *<1 byte value 1 type code>*
>>>> > >> > *<N byte value 1 value>*
>>>> > >> > *<1 byte key 2 **name **type code>*
>>>> > >> > *<N byte key 2 name data>*
>>>> > >> > *<1 byte value 2 type code>*
>>>> > >> > *<N byte value 2 value>*
>>>> > >> > etc
>>>> > >> >
>>>> > >> > Also that allow to put nested dictionaries without changes in
>>>> type
>>>> > code
>>>> > >> > table!
>>>> > >> > Of  course for the appoach above we get significat overhead for
>>>> key
>>>> > >> > storing. But I think it is acceptable for some cases and
>>>> definitely ok
>>>> > >> for
>>>> > >> > Python
>>>> > >> >
>>>> > >> >
>>>> > >> >
>>>> > >> >
>>>> > >> > On Wed, Nov 22, 2017 at 9:14 PM, Prachi Garg <pgarg@gridgain.com
>>>> >
>>>> > >> wrote:
>>>> > >> >
>>>> > >> > > Thanks Pavel! The document has good information. I'll create
>>>> one on
>>>> > >> > > readme.io; will also add some examples there.
>>>> > >> > >
>>>> > >> > > On Wed, Nov 22, 2017 at 5:03 AM, Pavel Tupitsyn <
>>>> > ptupitsyn@apache.org
>>>> > >> >
>>>> > >> > > wrote:
>>>> > >> > >
>>>> > >> > > > Igniters,
>>>> > >> > > >
>>>> > >> > > > I've put together a detailed description of our Thin Client
>>>> > protocol
>>>> > >> > > > in form of IEP on wiki:
>>>> > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
>>>> > >> > > > 9+Thin+Client+Protocol
>>>> > >> > > >
>>>> > >> > > >
>>>> > >> > > > To clarify:
>>>> > >> > > > - Protocol implementation is in master (see
>>>> ClientMessageParser
>>>> > >> class)
>>>> > >> > > > - Protocol has not been released yet, so we are free to
>>>> change
>>>> > >> anything
>>>> > >> > > > - Protocol is only used by .NET Thin Client for now, but is
>>>> > >> supposed to
>>>> > >> > > be
>>>> > >> > > > used from other languages by third party contributors
>>>> > >> > > > - More operations will be added in future, this is a first
>>>> set of
>>>> > >> them,
>>>> > >> > > > cache-related
>>>> > >> > > >
>>>> > >> > > >
>>>> > >> > > > Please review the document and let me know your thoughts.
>>>> > >> > > > Is there anything missing or wrong?
>>>> > >> > > >
>>>> > >> > > > We should make sure that the foundation is solid and
>>>> extensible.
>>>> > >> > > >
>>>> > >> > > >
>>>> > >> > > > Thanks,
>>>> > >> > > > Pavel
>>>> > >> > > >
>>>> > >> > >
>>>> > >> >
>>>> > >> >
>>>> > >> >
>>>> > >> > --
>>>> > >> > Sergey Kozlov
>>>> > >> > GridGain Systems
>>>> > >> > www.gridgain.com
>>>> > >> >
>>>> > >>
>>>> > >
>>>> > >
>>>> > >
>>>> > > --
>>>> > > Sergey Kozlov
>>>> > > GridGain Systems
>>>> > > www.gridgain.com
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Sergey Kozlov
>>>> > GridGain Systems
>>>> > www.gridgain.com
>>>> >
>>>>
>>>
>>>
>>
>

--089e0825888cd2dbf0055f2e660d--