Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
References: <1500988415143-20024.post@n4.nabble.com> <CA+Bp9OAADegY0wFkzUTqQ+eEUHUDw010HHo6rxpz_yE6GQM6cQ@mail.gmail.com>
 <CAGt6o8kig43+jhHOsyhO0h6qUQe0NStO-Yhj2q+xPDCvBhuJYQ@mail.gmail.com> <CA+0=VoXOr=+sgTSrfW4CCcdbpaz5sAYkcR=jY5rOUe-aoAj9iQ@mail.gmail.com>
In-Reply-To: <CA+0=VoXOr=+sgTSrfW4CCcdbpaz5sAYkcR=jY5rOUe-aoAj9iQ@mail.gmail.com>
From: Vladimir Ozerov <vozerov@gridgain.com>
Date: Tue, 25 Jul 2017 17:36:56 +0000
Message-ID: <CAGt6o8kMOz6MP+Kz0E-UyfrGmWBnN-QhUhrBqup3gjSGUkL19Q@mail.gmail.com>
Subject: Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary="001a114eea8ebee446055527c7bf"
archived-at: Tue, 25 Jul 2017 17:37:19 -0000

--001a114eea8ebee446055527c7bf
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Vyacheslav,
When we finish varlen optimization for string lengths, I am afraid we could
end up with very messy protocol, should we mix encoded length and encoding.

Dima,
Encoding must be set on per field basis. This will give us as most flexible
solution at the cost of 1-byte overhead.

=D0=B2=D1=82, 25 =D0=B8=D1=8E=D0=BB=D1=8F 2017 =D0=B3. =D0=B2 20:23, Dmitri=
y Setrakyan <dsetrakyan@apache.org>:

> I don't understand why this encoding is done on per-object and not on
> per-cache level. Shouldn't the column-to-encoding mapping be defined at
> cache level configuration?
>
> On Tue, Jul 25, 2017 at 12:13 PM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Andrey,
> >
> > You cannot have optional part in the middle as it will break
> compatibility
> > in dangerous way, probably leading to node crash. Also having INT (4
> bytes)
> > looks too much for me.
> >
> > Instead, I would add new type "encoded string":
> > 1 byte - type
> > 1 byte - encoding code, map frequently used encodings to some byte valu=
e;
> > also have a special value, meaning that encoding will be written as
> string
> > afterwards, this way we will support any encoding out of the box
> > [optional] encoding name
> > 4 bytes - string length
> > Finally - string bytes
> >
> > Vladimir.
> >
> > =D0=B2=D1=82, 25 =D0=B8=D1=8E=D0=BB=D1=8F 2017 =D0=B3. =D0=B2 18:24, An=
drey Kuznetsov <stkuzma@gmail.com>:
> >
> > > I apologize for damaged formatting. Below is my message as it should
> be.
> > >
> > >
> > > Hi Igniters,
> > >
> > > I'd like to discuss future changes related to
> https://issues.apache.org/
> > > jira/browse/IGNITE-5655
> > > <https://issues.apache.org/jira/browse/IGNITE-5655>.
> > >
> > > Is it really good idea to introduce new flag (ENCODED_STRING) for
> > existing
> > > String datatype? It's possible to use existing STRING flag at
> negligible
> > > performance cost.
> > >
> > > Currently, utf-8-encoded string looks like
> > >
> > > byteFlag nonNegativeIntStrLen bytes
> > >
> > > This format can be backward compatibly extended to
> > >
> > > byteFlag [negativeIntCharsetCode] nonNegativeIntStrLen bytes
> > >
> > > Next, I suggest to add new BinaryConfiguration property for encoding =
to
> > use
> > > instead of using global property. It seems to be more convenient for
> > user.
> > >
> > > I'll appreciate your feedback.
> > >
> > > 2017-07-25 16:13 GMT+03:00 Andrey Kuznetsov <stkuzma@gmail.com>:
> > >
> > > > Hi Igniters,I'd like to discuss future changes related to
> IGNITE-5655
> > > > <https://issues.apache.org/jira/browse/IGNITE-5655>  . Is it really
> > good
> > > > idea to introduce new flag (ENCODED_STRING) for existing String
> > datatype?
> > > > It's possible to use existing STRING flag at negligible performance
> > cost.
> > > > Currently, utf-8-encoded string looks like
> > > > byteFlag nonNegativeIntStrLen bytes
> > > > This format can be backward compatibly extended to
> > > > byteFlag [negativeIntCharsetCode] nonNegativeIntStrLen bytes
> > > > Next, I suggest to add new BinaryConfiguration property for encodin=
g
> to
> > > use
> > > > instead of using global property. It seems to be more convenient fo=
r
> > > > user.I'll appreciate your feedback.
> > > >
> > > >
> > > >
> > > > -----
> > > > Best regards,
> > > >   Andrey Kuznetsov.
> > > > --
> > > > View this message in context: http://apache-ignite-
> > > > developers.2346864.n4.nabble.com/Non-UTF-8-string-encoding-
> > > > support-in-BinaryMarshaller-IGNITE-5655-tp20024.html
> > > > Sent from the Apache Ignite Developers mailing list archive at
> > > Nabble.com.
> > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >   Andrey Kuznetsov.
> > >
> >
>

--001a114eea8ebee446055527c7bf--