ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergi Vladykin <sergi.vlady...@gmail.com>
Subject Re: Stable binary key representation
Date Sun, 09 Apr 2017 10:54:16 GMT
Ok, we need to do 2 things here:

1. Drop the resolvers from the source code.
2. Write a good page in docs on "What makes a correct cache key".

Who can do that?

Sergi

2017-04-07 9:48 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com>:

> It is possible to try adding support of comparison to Resolvers, but the
> whole approach looks wrong and for now it is better to get rid of it while
> we have a chance to break compatibility.
>
> Sergi
>
> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> valentin.kulichenko@gmail.com>:
>
>> The discussion should've been started with that :) If supporting resolvers
>> in new architecture is not possible or means too big effort, then it's
>> definitely not worth it.
>>
>> -Val
>>
>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vozerov@gridgain.com>
>> wrote:
>>
>> > Dima,
>> >
>> > Yes, they may explode some internals of our indexes.
>> >
>> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan" <
>> > dsetrakyan@apache.org> написал:
>> >
>> > > Guys,
>> > >
>> > > Isn't the main issue here that we cannot use the Identity Resolvers in
>> > > BTrees in the 2.0 version? If yes, then we have to remove them no
>> matter
>> > > what.
>> > >
>> > > D.
>> > >
>> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
>> sergi.vladykin@gmail.com
>> > >
>> > > wrote:
>> > >
>> > > > Binary key representation is stable when we always have equal
>> > serialized
>> > > > bytes when the original keys are equal.
>> > > >
>> > > > Resolver allows you to have some extra info in the Key and equal
>> Keys
>> > > will
>> > > > be serialized into different bytes, which is wrong.
>> > > >
>> > > > Look at the example what you can do with resolvers:
>> > > >
>> > > > We may have some data entry with fields a, b, c. Let's say the
>> unique
>> > > part
>> > > > here is `a` and it the only fields used in Key equals() and
>> hashCode().
>> > > > Still we may have the following layouts:
>> > > >
>> > > > 1. Ka -> Vbc
>> > > > 2. Kab -> Vc
>> > > > 3. Kabc -> Boolean.TRUE
>> > > >
>> > > > The only 1 is a correct layout, others are plain wrong variants (but
>> > they
>> > > > are still possible with Resolvers) because everything that does not
>> > make
>> > > > Key unique must be in Value.
>> > > >
>> > > > We want to clearly state that if you have something in Key, that is
>> not
>> > > > part of equals(), then the Key is invalid and that stuff must be in
>> > > Value.
>> > > > This allows us to rely on binary representation of a Key to be
>> stable
>> > and
>> > > > have some more optimizations and code simplifications with respect
>> to
>> > > these
>> > > > assumptions.
>> > > >
>> > > > Sergi
>> > > >
>> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
>> > > > valentin.kulichenko@gmail.com>:
>> > > >
>> > > > > Even with my vast expirience I would never claim that I've seen
>> > > > > "everything" :)
>> > > > >
>> > > > > What do you mean by stable binary key representation and how
>> > resolvers
>> > > > make
>> > > > > it unstable?
>> > > > >
>> > > > > -Val
>> > > > >
>> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
>> > > sergi.vladykin@gmail.com
>> > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Val,
>> > > > > >
>> > > > > > I know that you have really vast experience in Ignite
>> deployments
>> > and
>> > > > > > probably saw everything that can happen. Did you ever see
>> identity
>> > > > > > resolvers use in real life? I guess no.
>> > > > > >
>> > > > > > Hibernate example is bad here, because if their key is unstable
>> > > across
>> > > > > > multiple JVMs, it means that it was not designed for distributed
>> > > > caches a
>> > > > > > priori.
>> > > > > >
>> > > > > > Also knowing in advance about stable binary key representation
>> > allows
>> > > > us
>> > > > > to
>> > > > > > apply additional optimizations, like comparing keys without
>> > detaching
>> > > > > them
>> > > > > > from offheap memory.
>> > > > > >
>> > > > > > We always will be able to add this stuff back if we see
users
>> > really
>> > > > need
>> > > > > > it. Let's remove it for 2.0.
>> > > > > >
>> > > > > > Sergi
>> > > > > >
>> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko <
>> > > > > > valentin.kulichenko@gmail.com>:
>> > > > > >
>> > > > > > > Alex,
>> > > > > > >
>> > > > > > > To be honest, I don't understand the reasoning behind
the
>> > removal.
>> > > I
>> > > > > > think
>> > > > > > > resolvers provide good flexibility for different corner
cases
>> and
>> > > > it's
>> > > > > a
>> > > > > > > good thing to have them. Note that they can be applied
not
>> only
>> > to
>> > > > > cache
>> > > > > > > keys, but to any binary objects.
>> > > > > > >
>> > > > > > > Hibernate issue is actually a good example of such
use case.
>> The
>> > > fact
>> > > > > > that
>> > > > > > > we found an alternative solution doesn't actually mean
>> anything,
>> > > > > because
>> > > > > > > what if this happened not in our module, but in user's
>> > application?
>> > > > > > > Unfortunately, we can't predict everything.
>> > > > > > >
>> > > > > > > Error proneness is not a very strong argument either,
because
>> in
>> > my
>> > > > > view
>> > > > > > > these resolvers are as much error prone as BinaryIdMapper,
for
>> > > > example.
>> > > > > > >
>> > > > > > > -Val
>> > > > > > >
>> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk <
>> > > > > > > alexey.goncharuk@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Denis,
>> > > > > > > >
>> > > > > > > > Can you suggest a use-case where identity resolver
is needed
>> > > (given
>> > > > > > that
>> > > > > > > we
>> > > > > > > > agree that a key must contain only valuable fields)?
>> > > > > > > >
>> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda <dmagda@apache.org>:
>> > > > > > > >
>> > > > > > > > > Where do you want to remove the identity
resolvers from?
>> If
>> > > it’s
>> > > > > > > related
>> > > > > > > > > to the internals of Hibernate module then
it’s fine but if
>> > you
>> > > > > > suggest
>> > > > > > > > > removing identity resolvers public interfaces
then it
>> might
>> > be
>> > > a
>> > > > > > haste
>> > > > > > > > > decision.
>> > > > > > > > >
>> > > > > > > > > —
>> > > > > > > > > Denis
>> > > > > > > > >
>> > > > > > > > > > On Apr 5, 2017, at 7:42 AM, Alexey Goncharuk
<
>> > > > > > > > alexey.goncharuk@gmail.com>
>> > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > +1, I see no other reasons to keep it.
>> > > > > > > > > >
>> > > > > > > > > > 2017-04-05 13:59 GMT+03:00 Sergi Vladykin
<
>> > > > > > sergi.vladykin@gmail.com
>> > > > > > > >:
>> > > > > > > > > >
>> > > > > > > > > >> +1
>> > > > > > > > > >>
>> > > > > > > > > >> Lets drop them.
>> > > > > > > > > >>
>> > > > > > > > > >> Sergi
>> > > > > > > > > >>
>> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00 Dmitriy
Govorukhin <
>> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
>> > > > > > > > > >> :
>> > > > > > > > > >>
>> > > > > > > > > >>> Hi guys, i implemented proxy
for IgniteCache in
>> hibernate
>> > > > > > > > integration,
>> > > > > > > > > >> this
>> > > > > > > > > >>> proxy transformate cacheKey
to our key wrapper, leaves
>> > only
>> > > > > > > required
>> > > > > > > > > >>> field. I think we can remove
identity resolve, it
>> should
>> > > not
>> > > > > > broke
>> > > > > > > > > >>> integration with hibernate.
Any objections?
>> > > > > > > > > >>>
>> > > > > > > > > >>> On Wed, Mar 29, 2017 at 11:07
PM, Valentin Kulichenko
>> <
>> > > > > > > > > >>> valentin.kulichenko@gmail.com>
wrote:
>> > > > > > > > > >>>
>> > > > > > > > > >>>> I'm not saying there is
no alternative solution. But
>> > let's
>> > > > > > > implement
>> > > > > > > > > it
>> > > > > > > > > >>> and
>> > > > > > > > > >>>> prove that it works first,
and remove resolvers only
>> > after
>> > > > > that.
>> > > > > > > > > >>>>
>> > > > > > > > > >>>> -Val
>> > > > > > > > > >>>>
>> > > > > > > > > >>>> On Wed, Mar 29, 2017 at
12:18 PM, Sergi Vladykin <
>> > > > > > > > > >>> sergi.vladykin@gmail.com
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>> wrote:
>> > > > > > > > > >>>>
>> > > > > > > > > >>>>> Guys, nothing is impossible
if you know a bit about
>> > > > > reflection
>> > > > > > in
>> > > > > > > > > >> Java
>> > > > > > > > > >>> :)
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> We had a look at the
CacheKey class and it is easily
>> > > > > > replaceable.
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> Sergi
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>> 2017-03-29 21:49 GMT+03:00
Dmitriy Setrakyan <
>> > > > > > > > dsetrakyan@apache.org
>> > > > > > > > > >>> :
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>>> On Wed, Mar 29,
2017 at 11:43 AM, Valentin
>> Kulichenko
>> > <
>> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com>
wrote:
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>>> "Hibernate key"
is the CacheKey class I was
>> referring
>> > > to.
>> > > > > > It's
>> > > > > > > > > >>>> provided
>> > > > > > > > > >>>>>> by
>> > > > > > > > > >>>>>>> Hibernate, not
by user and not by us. So I'm not
>> sure
>> > > > it's
>> > > > > > > > > >> possible
>> > > > > > > > > >>>> to
>> > > > > > > > > >>>>>>> replace it.
>> > > > > > > > > >>>>>>>
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>> If it is impossible
to replace or get rid of the
>> > > Hibernate
>> > > > > > key,
>> > > > > > > is
>> > > > > > > > > >>> this
>> > > > > > > > > >>>>>> discussion valid
at all?
>> > > > > > > > > >>>>>>
>> > > > > > > > > >>>>>
>> > > > > > > > > >>>>
>> > > > > > > > > >>>
>> > > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message