Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CAK2_iMMU0SPA9FZQcWPBrmPSFUSZpUdJw3VB+SmKQuU2NPOw+Q@mail.gmail.com>
References: <CAFyTW-jBaFz-Twe7hy9-5Vigz0OZn6i5-Crq224bDbSvj-+WYA@mail.gmail.com>
 <CA+0=VoWU88bXW87k7amPyu7t76TCo-fODJOQf8yKDRexDcm7Vg@mail.gmail.com>
 <CALOGAoxj3G3r4vgAfOaTbh6rODo+weWSuxYtyK1i+8nZk-otjQ@mail.gmail.com>
 <CA+0=VoV8tKQx-gMwnAf_gKaWh--hgCKVDrJLt+5y6MAZSNBsyw@mail.gmail.com>
 <CAK2_iMPysa5LCHwgN3RcEu4XjS9uXTioDGgpG9dM1gxuAtQc3w@mail.gmail.com>
 <CA+0=VoWij19Xe33DMUS+AAoVauey6ATF+5Q+O_aQsdoi-1n5Tg@mail.gmail.com>
 <CAK2_iMMNphdrO7iJMh5CnpAD6_71hJwOoYvVC2hqtxhdg4d+mQ@mail.gmail.com>
 <CABuYRcrG6yPQr2BgcLC5tEGrrKXhy15-aSpEzgEn1C4gWpZEkg@mail.gmail.com>
 <CAK2_iMMQ4dHP8iUdUzGMHiGVbiSgp6NxtgQK17fdddTWBts6BQ@mail.gmail.com>
 <CAGt6o8=EN0Wuv91PrHys=9-59_t1XcN2i+GHd9j0eSNO1VZ3Og@mail.gmail.com>
 <CA+Bp9OA6qC+4hnHb1=MKUv-Bo+XQA=cbovLJUymGb_+ai0FsaQ@mail.gmail.com>
 <CAGt6o8n-UwGuRrKSRHqKcWzfbahv7NbR19f9Oa6tTTN5O84dig@mail.gmail.com> <CAK2_iMMU0SPA9FZQcWPBrmPSFUSZpUdJw3VB+SmKQuU2NPOw+Q@mail.gmail.com>
From: Vladimir Ozerov <vozerov@gridgain.com>
Date: Fri, 28 Jul 2017 14:45:33 +0300
Message-ID: <CAGt6o8nREurLnz2iXGBE87r887-N-oTAR0WY_atG7paskc75CA@mail.gmail.com>
Subject: Re: Non-UTF-8 string encoding support in BinaryMarshaller (IGNITE-5655)
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary="f4030437979011067c05555f3889"
archived-at: Fri, 28 Jul 2017 11:45:41 -0000

--f4030437979011067c05555f3889
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

String encoding is a concept similar to "collation" in RDBMS. You can
define it either globally, or on per-table basis. The same should be done
for Ignite. We do not define behavior of a type. We define behavior of a
*storage*.

Two cases when proposed approach with per-type and per-type-field approach
doesn't work:
1) I have a class Person with field "name". I have two caches/tables - one
for US persons, where name is in Latin, another for RU persons with
Cyrillic names. How can achieve optimal encoding formats for both tables?
2) I have an empty grid. Now I want to create a cache/table with custom
encoding. How can I do that without cluster restart? Nohow, because
BinaryTypeConfiguration configured statically, while caches/tables can be
created in runtime.

On Fri, Jul 28, 2017 at 2:38 PM, Pavel Tupitsyn <ptupitsyn@apache.org>
wrote:

> > As Pavel mentioned, Marshaller should not be tied to cache
> > should be added to per-cache level
> Not sure if I follow.
> Marshalling and caching are two separate mechanisms.
> Defining binary format in CacheConfiguration violates separation of
> concerns.
>
> > Encoding *must not* be added to per-class or per-field level, this is
> wrong
> What is wrong with this? BinaryTypeConfiguration looks the right place fo=
r
> such a setting.
> Are we talking from SQL standpoint here, so you want this to be defined
> somehow via DDL in future?
>
> On Fri, Jul 28, 2017 at 2:30 PM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
>
> > Encoding *must not* be added to per-class or per-field level, this is
> > wrong.
> >
> > It should be added to per-cache level, and to per-cache-column level in
> > future.
> >
> > =D0=BF=D1=82, 28 =D0=B8=D1=8E=D0=BB=D1=8F 2017 =D0=B3. =D0=B2 14:27, An=
drey Kuznetsov <stkuzma@gmail.com>:
> >
> > > We discussed this with Pavel and Anton just a moment ago. Summary
> > follows.
> > >
> > > - New byte "flag" is to be added (ENCODED_STRING)
> > > - 'Encoding' property is to be added at
> > >   -- global level (BinaryConfiguration)
> > >   -- per-class level (BinaryTypeConfiguration)
> > >   -- per-field level (BinaryTypeConfiguration)
> > >
> > > 2017-07-28 14:15 GMT+03:00 Vladimir Ozerov [via Apache Ignite
> > Developers] <
> > > ml+s2346864n20159h78@n4.nabble.com>:
> > >
> > > > As Pavel mentioned, Marshaller should not be tied to cache,
> > BinaryObject
> > > > should be self-explanatory, i.e. containing all information necessa=
ry
> > for
> > > > unmarshalling. This is an absolute requirement.
> > > >
> > > > We will have one extra byte for in serialized form, meaning that
> > > advantage
> > > > of custom encoding will become evident for all strings with length =
>=3D
> > 1,
> > > > which is perfectly fine. I do not quite understand what are we
> arguing
> > > > about.
> > > >
> > > > As far as configuration, we can do it as follows:
> > > >
> > > > 1) Add global encoding, UTF8 by default.
> > > > 2) Add per-cache encoding.
> > > > 3) Add encoding to JDBC and ODBC driver properties.
> > > >
> > > > This should be enough.
> > > >
> > > >
> > > --
> > > Best regards,
> > >   Andrey Kuznetsov.
> > >
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://apache-ignite-developers.2346864.n4.nabble.
> > com/Non-UTF-8-string-encoding-support-in-BinaryMarshaller-
> > IGNITE-5655-tp20024p20161.html
> > > Sent from the Apache Ignite Developers mailing list archive at
> > Nabble.com.
> >
>

--f4030437979011067c05555f3889--