ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)
Date Tue, 25 Jul 2017 17:36:56 GMT
Vyacheslav,
When we finish varlen optimization for string lengths, I am afraid we could
end up with very messy protocol, should we mix encoded length and encoding.

Dima,
Encoding must be set on per field basis. This will give us as most flexible
solution at the cost of 1-byte overhead.

вт, 25 июля 2017 г. в 20:23, Dmitriy Setrakyan <dsetrakyan@apache.org>:

> I don't understand why this encoding is done on per-object and not on
> per-cache level. Shouldn't the column-to-encoding mapping be defined at
> cache level configuration?
>
> On Tue, Jul 25, 2017 at 12:13 PM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Andrey,
> >
> > You cannot have optional part in the middle as it will break
> compatibility
> > in dangerous way, probably leading to node crash. Also having INT (4
> bytes)
> > looks too much for me.
> >
> > Instead, I would add new type "encoded string":
> > 1 byte - type
> > 1 byte - encoding code, map frequently used encodings to some byte value;
> > also have a special value, meaning that encoding will be written as
> string
> > afterwards, this way we will support any encoding out of the box
> > [optional] encoding name
> > 4 bytes - string length
> > Finally - string bytes
> >
> > Vladimir.
> >
> > вт, 25 июля 2017 г. в 18:24, Andrey Kuznetsov <stkuzma@gmail.com>:
> >
> > > I apologize for damaged formatting. Below is my message as it should
> be.
> > >
> > >
> > > Hi Igniters,
> > >
> > > I'd like to discuss future changes related to
> https://issues.apache.org/
> > > jira/browse/IGNITE-5655
> > > <https://issues.apache.org/jira/browse/IGNITE-5655>.
> > >
> > > Is it really good idea to introduce new flag (ENCODED_STRING) for
> > existing
> > > String datatype? It's possible to use existing STRING flag at
> negligible
> > > performance cost.
> > >
> > > Currently, utf-8-encoded string looks like
> > >
> > > byteFlag nonNegativeIntStrLen bytes
> > >
> > > This format can be backward compatibly extended to
> > >
> > > byteFlag [negativeIntCharsetCode] nonNegativeIntStrLen bytes
> > >
> > > Next, I suggest to add new BinaryConfiguration property for encoding to
> > use
> > > instead of using global property. It seems to be more convenient for
> > user.
> > >
> > > I'll appreciate your feedback.
> > >
> > > 2017-07-25 16:13 GMT+03:00 Andrey Kuznetsov <stkuzma@gmail.com>:
> > >
> > > > Hi Igniters,I'd like to discuss future changes related to
> IGNITE-5655
> > > > <https://issues.apache.org/jira/browse/IGNITE-5655>  . Is it really
> > good
> > > > idea to introduce new flag (ENCODED_STRING) for existing String
> > datatype?
> > > > It's possible to use existing STRING flag at negligible performance
> > cost.
> > > > Currently, utf-8-encoded string looks like
> > > > byteFlag nonNegativeIntStrLen bytes
> > > > This format can be backward compatibly extended to
> > > > byteFlag [negativeIntCharsetCode] nonNegativeIntStrLen bytes
> > > > Next, I suggest to add new BinaryConfiguration property for encoding
> to
> > > use
> > > > instead of using global property. It seems to be more convenient for
> > > > user.I'll appreciate your feedback.
> > > >
> > > >
> > > >
> > > > -----
> > > > Best regards,
> > > >   Andrey Kuznetsov.
> > > > --
> > > > View this message in context: http://apache-ignite-
> > > > developers.2346864.n4.nabble.com/Non-UTF-8-string-encoding-
> > > > support-in-BinaryMarshaller-IGNITE-5655-tp20024.html
> > > > Sent from the Apache Ignite Developers mailing list archive at
> > > Nabble.com.
> > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >   Andrey Kuznetsov.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message