Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CA+eZwrF_-n8WxLhr92_+m8SMjSws3GQ=pupcgZNJ5x-iM5Mz3Q@mail.gmail.com>
References: <CABDss3gAsqL8NmTUPuip7iwv51BvgxQ0T3g_xqu+_y6AQwnshA@mail.gmail.com>
 <CA+0=VoW=v=La6QimMv_XGUZwpKKmGGfMuaxHDeN-mBtZ6cOwYQ@mail.gmail.com>
 <CABuYRcoEVYNOw5Va82CkTdi6XRN8D_y+B7ToGOwZ_fV7=w3Tzg@mail.gmail.com>
 <CA+0=VoU2DeeAyF0LCyRD0OGEbyVcsdUd6XHRBuEiYjU1e5tj3g@mail.gmail.com>
 <CA+eZwrHmo+r13zHxGK66LZA_qt+HwsYJWrpTy8g_Rqhzep0cww@mail.gmail.com>
 <CABuYRcpCcxfx0EdSdZry19uW4KBxX74wuWNwAPUOpJf3ZsoKVQ@mail.gmail.com>
 <CAPzXPXouLsdMvpj2sxR1mKXxuOXGG9rLhoXV06_f1fNPUBEQ6w@mail.gmail.com>
 <CA+eZwrETqoVJtL4+C9kjeixk0uqO+TDrVCqMhMA_AaMqWGBDBw@mail.gmail.com>
 <CABDss3iCHC8YeBdqNpo67Tx19AAfDJ3ezfK5Atk2As5PnJ6bTw@mail.gmail.com>
 <5D9118AC-CAE9-47FC-92CC-339AC5A86675@apache.org> <CABDss3j6yiVxHwj6Hai4zOgpX79sH0=4-o2H+OfcE09r6zu6Tg@mail.gmail.com>
 <CABuYRcopbK0YBxWYfxAofRZwjaPQnkRQiJjZR9QynU9MnDRKuQ@mail.gmail.com>
 <CA+eZwrGn1HmgXGFYW0ME3hLM7bvJv6z1JkSX9gwicaBWPsA_jg@mail.gmail.com>
 <CABuYRcr+B7N6uJs0otTEiKei-zy-Pzyncpe+Q4CSYxp8O076oA@mail.gmail.com>
 <CA+eZwrGRsXmfeyLKGr=LnJCeG0P+xiRKFXGrqHOO7RfJvonBOQ@mail.gmail.com>
 <CA+0=VoV-Qv60v8L2sKMRCdWr_bjNPEQ1JRZj0hLtbi6gwERNbw@mail.gmail.com>
 <CAGt6o8k3H6o1j6HUzXdzK-u94oSqZk4-iBFTgVDxZaRxV_B8Kw@mail.gmail.com>
 <CABuYRcqiBUMmp83TwNNCBs-6o6fFKY=g5_kX1shLWH=boPe-fw@mail.gmail.com> <CA+eZwrF_-n8WxLhr92_+m8SMjSws3GQ=pupcgZNJ5x-iM5Mz3Q@mail.gmail.com>
From: Sergi Vladykin <sergi.vladykin@gmail.com>
Date: Sun, 9 Apr 2017 13:54:16 +0300
Message-ID: <CA+eZwrH9GD98uMFq2bFLtFsKyohsze8+HMbjweOfmKXueiBvdA@mail.gmail.com>
Subject: Re: Stable binary key representation
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary=f403045c01fe750400054cb9b009
archived-at: Sun, 09 Apr 2017 10:55:04 -0000

--f403045c01fe750400054cb9b009
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Ok, we need to do 2 things here:

1. Drop the resolvers from the source code.
2. Write a good page in docs on "What makes a correct cache key".

Who can do that?

Sergi

2017-04-07 9:48 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com>:

> It is possible to try adding support of comparison to Resolvers, but the
> whole approach looks wrong and for now it is better to get rid of it whil=
e
> we have a chance to break compatibility.
>
> Sergi
>
> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
>> The discussion should've been started with that :) If supporting resolve=
rs
>> in new architecture is not possible or means too big effort, then it's
>> definitely not worth it.
>>
>> -Val
>>
>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vozerov@gridgain.com>
>> wrote:
>>
>> > Dima,
>> >
>> > Yes, they may explode some internals of our indexes.
>> >
>> > 06 =D0=B0=D0=BF=D1=80. 2017 =D0=B3. 23:32 =D0=BF=D0=BE=D0=BB=D1=8C=D0=
=B7=D0=BE=D0=B2=D0=B0=D1=82=D0=B5=D0=BB=D1=8C "Dmitriy Setrakyan" <
>> > dsetrakyan@apache.org> =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB:
>> >
>> > > Guys,
>> > >
>> > > Isn't the main issue here that we cannot use the Identity Resolvers =
in
>> > > BTrees in the 2.0 version? If yes, then we have to remove them no
>> matter
>> > > what.
>> > >
>> > > D.
>> > >
>> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
>> sergi.vladykin@gmail.com
>> > >
>> > > wrote:
>> > >
>> > > > Binary key representation is stable when we always have equal
>> > serialized
>> > > > bytes when the original keys are equal.
>> > > >
>> > > > Resolver allows you to have some extra info in the Key and equal
>> Keys
>> > > will
>> > > > be serialized into different bytes, which is wrong.
>> > > >
>> > > > Look at the example what you can do with resolvers:
>> > > >
>> > > > We may have some data entry with fields a, b, c. Let's say the
>> unique
>> > > part
>> > > > here is `a` and it the only fields used in Key equals() and
>> hashCode().
>> > > > Still we may have the following layouts:
>> > > >
>> > > > 1. Ka -> Vbc
>> > > > 2. Kab -> Vc
>> > > > 3. Kabc -> Boolean.TRUE
>> > > >
>> > > > The only 1 is a correct layout, others are plain wrong variants (b=
ut
>> > they
>> > > > are still possible with Resolvers) because everything that does no=
t
>> > make
>> > > > Key unique must be in Value.
>> > > >
>> > > > We want to clearly state that if you have something in Key, that i=
s
>> not
>> > > > part of equals(), then the Key is invalid and that stuff must be i=
n
>> > > Value.
>> > > > This allows us to rely on binary representation of a Key to be
>> stable
>> > and
>> > > > have some more optimizations and code simplifications with respect
>> to
>> > > these
>> > > > assumptions.
>> > > >
>> > > > Sergi
>> > > >
>> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
>> > > > valentin.kulichenko@gmail.com>:
>> > > >
>> > > > > Even with my vast expirience I would never claim that I've seen
>> > > > > "everything" :)
>> > > > >
>> > > > > What do you mean by stable binary key representation and how
>> > resolvers
>> > > > make
>> > > > > it unstable?
>> > > > >
>> > > > > -Val
>> > > > >
>> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
>> > > sergi.vladykin@gmail.com
>> > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Val,
>> > > > > >
>> > > > > > I know that you have really vast experience in Ignite
>> deployments
>> > and
>> > > > > > probably saw everything that can happen. Did you ever see
>> identity
>> > > > > > resolvers use in real life? I guess no.
>> > > > > >
>> > > > > > Hibernate example is bad here, because if their key is unstabl=
e
>> > > across
>> > > > > > multiple JVMs, it means that it was not designed for distribut=
ed
>> > > > caches a
>> > > > > > priori.
>> > > > > >
>> > > > > > Also knowing in advance about stable binary key representation
>> > allows
>> > > > us
>> > > > > to
>> > > > > > apply additional optimizations, like comparing keys without
>> > detaching
>> > > > > them
>> > > > > > from offheap memory.
>> > > > > >
>> > > > > > We always will be able to add this stuff back if we see users
>> > really
>> > > > need
>> > > > > > it. Let's remove it for 2.0.
>> > > > > >
>> > > > > > Sergi
>> > > > > >
>> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
>> > > > > > valentin.kulichenko@gmail.com>:
>> > > > > >
>> > > > > > > Alex,
>> > > > > > >
>> > > > > > > To be honest, I don't understand the reasoning behind the
>> > removal.
>> > > I
>> > > > > > think
>> > > > > > > resolvers provide good flexibility for different corner case=
s
>> and
>> > > > it's
>> > > > > a
>> > > > > > > good thing to have them. Note that they can be applied not
>> only
>> > to
>> > > > > cache
>> > > > > > > keys, but to any binary objects.
>> > > > > > >
>> > > > > > > Hibernate issue is actually a good example of such use case.
>> The
>> > > fact
>> > > > > > that
>> > > > > > > we found an alternative solution doesn't actually mean
>> anything,
>> > > > > because
>> > > > > > > what if this happened not in our module, but in user's
>> > application?
>> > > > > > > Unfortunately, we can't predict everything.
>> > > > > > >
>> > > > > > > Error proneness is not a very strong argument either, becaus=
e
>> in
>> > my
>> > > > > view
>> > > > > > > these resolvers are as much error prone as BinaryIdMapper, f=
or
>> > > > example.
>> > > > > > >
>> > > > > > > -Val
>> > > > > > >
>> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
>> > > > > > > alexey.goncharuk@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Denis,
>> > > > > > > >
>> > > > > > > > Can you suggest a use-case where identity resolver is need=
ed
>> > > (given
>> > > > > > that
>> > > > > > > we
>> > > > > > > > agree that a key must contain only valuable fields)?
>> > > > > > > >
>> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dmagda@apache.org>=
:
>> > > > > > > >
>> > > > > > > > > Where do you want to remove the identity resolvers from?
>> If
>> > > it=E2=80=99s
>> > > > > > > related
>> > > > > > > > > to the internals of Hibernate module then it=E2=80=99s f=
ine but if
>> > you
>> > > > > > suggest
>> > > > > > > > > removing identity resolvers public interfaces then it
>> might
>> > be
>> > > a
>> > > > > > haste
>> > > > > > > > > decision.
>> > > > > > > > >
>> > > > > > > > > =E2=80=94
>> > > > > > > > > Denis
>> > > > > > > > >
>> > > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk <
>> > > > > > > > alexey.goncharuk@gmail.com>
>> > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > +1, I see no other reasons to keep it.
>> > > > > > > > > >
>> > > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin <
>> > > > > > sergi.vladykin@gmail.com
>> > > > > > > >:
>> > > > > > > > > >
>> > > > > > > > > >> +1
>> > > > > > > > > >>
>> > > > > > > > > >> Lets drop them.
>> > > > > > > > > >>
>> > > > > > > > > >> Sergi
>> > > > > > > > > >>
>> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy Govorukhin <
>> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
>> > > > > > > > > >> :
>> > > > > > > > > >>
>> > > > > > > > > >>> Hi guys, i implemented proxy for IgniteCache in
>> hibernate
>> > > > > > > > integration,
>> > > > > > > > > >> this
>> > > > > > > > > >>> proxy transformate cacheKey to our key wrapper, leav=
es
>> > only
>> > > > > > > required
>> > > > > > > > > >>> field. I think we can remove identity resolve, it
>> should
>> > > not
>> > > > > > broke
>> > > > > > > > > >>> integration with hibernate. Any objections?
>> > > > > > > > > >>>
>> > > > > > > > > >>> On Wed, Mar 29, 2017 at 11:07 PM, Valentin Kulichenk=
o
>> <
>> > > > > > > > > >>> valentin.kulichenko@gmail.com> wrote:
>> > > > > > > > > >>>
>> > > > > > > > > >>>> I'm not saying there is no alternative solution. Bu=
t
>> > let's
>> > > > > > > implement
>> > > > > > > > > it
>> > > > > > > > > >>> and
>> > > > > > > > > >>>> prove that it works first, and remove resolvers onl=
y
>> > after
>> > > > > that.
>> > > > > > > > > >>>>
>> > > > > > > > > >>>> -Val
>> > > > > > > > > >>>>
>> > > > > > > > > >>>> On Wed, Mar 29, 2017 at 12:18 PM, Sergi Vladykin <
>> > > > > > > > > >>> sergi.vladykin@gmail.com
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>> wrote:
>> > > > > > > > > >>>>
>> > > > > > > > > >>>>> Guys, nothing is impossible if you know a bit abou=
t
>> > > > > reflection
>> > > > > > in
>> > > > > > > > > >> Java
>> > > > > > > > > >>> :)
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> We had a look at the CacheKey class and it is easi=
ly
>> > > > > > replaceable.
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> Sergi
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
>> > > > > > > > dsetrakyan@apache.org
>> > > > > > > > > >>> :
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>> On Wed, Mar 29, 2017 at 11:43 AM, Valentin
>> Kulichenko
>> > <
>> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com> wrote:
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>>> "Hibernate key" is the CacheKey class I was
>> referring
>> > > to.
>> > > > > > It's
>> > > > > > > > > >>>> provided
>> > > > > > > > > >>>>>> by
>> > > > > > > > > >>>>>>> Hibernate, not by user and not by us. So I'm not
>> sure
>> > > > it's
>> > > > > > > > > >> possible
>> > > > > > > > > >>>> to
>> > > > > > > > > >>>>>>> replace it.
>> > > > > > > > > >>>>>>>
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>> If it is impossible to replace or get rid of the
>> > > Hibernate
>> > > > > > key,
>> > > > > > > is
>> > > > > > > > > >>> this
>> > > > > > > > > >>>>>> discussion valid at all?
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>
>> > > > > > > > > >>>
>> > > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

--f403045c01fe750400054cb9b009--