ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergi Vladykin <sergi.vlady...@gmail.com>
Subject Re: Stable binary key representation
Date Sun, 09 Apr 2017 14:30:16 GMT
I guess Resolvers were added to DML just because they already existed since
1.9 and we were forced to support them in all the parts of our product.

We have to stop this practice to add features without clear real life use
cases for them.

Sergi

2017-04-09 17:00 GMT+03:00 Denis Magda <dmagda@gridgain.com>:

> Sergi, Vovan,
>
> Sorry for being annoying but I still didn't get an answer on whether the
> resolvers are the must for DML. The main reason why we made them up some
> time ago is to support specific DML use cases. However I can't recall the
> use cases.
>
> --
> Denis
>
> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <sergi.vladykin@gmail.com>
> wrote:
>
> > Ok, we need to do 2 things here:
> >
> > 1. Drop the resolvers from the source code.
> > 2. Write a good page in docs on "What makes a correct cache key".
> >
> > Who can do that?
> >
> > Sergi
> >
> > 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com>:
> >
> > > It is possible to try adding support of comparison to Resolvers, but
> the
> > > whole approach looks wrong and for now it is better to get rid of it
> > while
> > > we have a chance to break compatibility.
> > >
> > > Sergi
> > >
> > > 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> > > valentin.kulichenko@gmail.com>:
> > >
> > >> The discussion should've been started with that :) If supporting
> > resolvers
> > >> in new architecture is not possible or means too big effort, then it's
> > >> definitely not worth it.
> > >>
> > >> -Val
> > >>
> > >> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <vozerov@gridgain.com
> >
> > >> wrote:
> > >>
> > >> > Dima,
> > >> >
> > >> > Yes, they may explode some internals of our indexes.
> > >> >
> > >> > 06 апр. 2017 г. 23:32 пользователь "Dmitriy Setrakyan"
<
> > >> > dsetrakyan@apache.org> написал:
> > >> >
> > >> > > Guys,
> > >> > >
> > >> > > Isn't the main issue here that we cannot use the Identity
> Resolvers
> > in
> > >> > > BTrees in the 2.0 version? If yes, then we have to remove them
no
> > >> matter
> > >> > > what.
> > >> > >
> > >> > > D.
> > >> > >
> > >> > > On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin <
> > >> sergi.vladykin@gmail.com
> > >> > >
> > >> > > wrote:
> > >> > >
> > >> > > > Binary key representation is stable when we always have
equal
> > >> > serialized
> > >> > > > bytes when the original keys are equal.
> > >> > > >
> > >> > > > Resolver allows you to have some extra info in the Key and
equal
> > >> Keys
> > >> > > will
> > >> > > > be serialized into different bytes, which is wrong.
> > >> > > >
> > >> > > > Look at the example what you can do with resolvers:
> > >> > > >
> > >> > > > We may have some data entry with fields a, b, c. Let's say
the
> > >> unique
> > >> > > part
> > >> > > > here is `a` and it the only fields used in Key equals()
and
> > >> hashCode().
> > >> > > > Still we may have the following layouts:
> > >> > > >
> > >> > > > 1. Ka -> Vbc
> > >> > > > 2. Kab -> Vc
> > >> > > > 3. Kabc -> Boolean.TRUE
> > >> > > >
> > >> > > > The only 1 is a correct layout, others are plain wrong variants
> > (but
> > >> > they
> > >> > > > are still possible with Resolvers) because everything that
does
> > not
> > >> > make
> > >> > > > Key unique must be in Value.
> > >> > > >
> > >> > > > We want to clearly state that if you have something in Key,
that
> > is
> > >> not
> > >> > > > part of equals(), then the Key is invalid and that stuff
must be
> > in
> > >> > > Value.
> > >> > > > This allows us to rely on binary representation of a Key
to be
> > >> stable
> > >> > and
> > >> > > > have some more optimizations and code simplifications with
> respect
> > >> to
> > >> > > these
> > >> > > > assumptions.
> > >> > > >
> > >> > > > Sergi
> > >> > > >
> > >> > > > 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko <
> > >> > > > valentin.kulichenko@gmail.com>:
> > >> > > >
> > >> > > > > Even with my vast expirience I would never claim that
I've
> seen
> > >> > > > > "everything" :)
> > >> > > > >
> > >> > > > > What do you mean by stable binary key representation
and how
> > >> > resolvers
> > >> > > > make
> > >> > > > > it unstable?
> > >> > > > >
> > >> > > > > -Val
> > >> > > > >
> > >> > > > > On Thu, Apr 6, 2017 at 2:36 AM, Sergi Vladykin <
> > >> > > sergi.vladykin@gmail.com
> > >> > > > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Val,
> > >> > > > > >
> > >> > > > > > I know that you have really vast experience in
Ignite
> > >> deployments
> > >> > and
> > >> > > > > > probably saw everything that can happen. Did you
ever see
> > >> identity
> > >> > > > > > resolvers use in real life? I guess no.
> > >> > > > > >
> > >> > > > > > Hibernate example is bad here, because if their
key is
> > unstable
> > >> > > across
> > >> > > > > > multiple JVMs, it means that it was not designed
for
> > distributed
> > >> > > > caches a
> > >> > > > > > priori.
> > >> > > > > >
> > >> > > > > > Also knowing in advance about stable binary key
> representation
> > >> > allows
> > >> > > > us
> > >> > > > > to
> > >> > > > > > apply additional optimizations, like comparing
keys without
> > >> > detaching
> > >> > > > > them
> > >> > > > > > from offheap memory.
> > >> > > > > >
> > >> > > > > > We always will be able to add this stuff back
if we see
> users
> > >> > really
> > >> > > > need
> > >> > > > > > it. Let's remove it for 2.0.
> > >> > > > > >
> > >> > > > > > Sergi
> > >> > > > > >
> > >> > > > > > 2017-04-06 11:21 GMT+03:00 Valentin Kulichenko
<
> > >> > > > > > valentin.kulichenko@gmail.com>:
> > >> > > > > >
> > >> > > > > > > Alex,
> > >> > > > > > >
> > >> > > > > > > To be honest, I don't understand the reasoning
behind the
> > >> > removal.
> > >> > > I
> > >> > > > > > think
> > >> > > > > > > resolvers provide good flexibility for different
corner
> > cases
> > >> and
> > >> > > > it's
> > >> > > > > a
> > >> > > > > > > good thing to have them. Note that they can
be applied not
> > >> only
> > >> > to
> > >> > > > > cache
> > >> > > > > > > keys, but to any binary objects.
> > >> > > > > > >
> > >> > > > > > > Hibernate issue is actually a good example
of such use
> case.
> > >> The
> > >> > > fact
> > >> > > > > > that
> > >> > > > > > > we found an alternative solution doesn't
actually mean
> > >> anything,
> > >> > > > > because
> > >> > > > > > > what if this happened not in our module,
but in user's
> > >> > application?
> > >> > > > > > > Unfortunately, we can't predict everything.
> > >> > > > > > >
> > >> > > > > > > Error proneness is not a very strong argument
either,
> > because
> > >> in
> > >> > my
> > >> > > > > view
> > >> > > > > > > these resolvers are as much error prone as
BinaryIdMapper,
> > for
> > >> > > > example.
> > >> > > > > > >
> > >> > > > > > > -Val
> > >> > > > > > >
> > >> > > > > > > On Wed, Apr 5, 2017 at 11:44 PM, Alexey Goncharuk
<
> > >> > > > > > > alexey.goncharuk@gmail.com> wrote:
> > >> > > > > > >
> > >> > > > > > > > Denis,
> > >> > > > > > > >
> > >> > > > > > > > Can you suggest a use-case where identity
resolver is
> > needed
> > >> > > (given
> > >> > > > > > that
> > >> > > > > > > we
> > >> > > > > > > > agree that a key must contain only valuable
fields)?
> > >> > > > > > > >
> > >> > > > > > > > 2017-04-05 22:08 GMT+03:00 Denis Magda
<
> dmagda@apache.org
> > >:
> > >> > > > > > > >
> > >> > > > > > > > > Where do you want to remove the
identity resolvers
> from?
> > >> If
> > >> > > it’s
> > >> > > > > > > related
> > >> > > > > > > > > to the internals of Hibernate module
then it’s fine
> but
> > if
> > >> > you
> > >> > > > > > suggest
> > >> > > > > > > > > removing identity resolvers public
interfaces then it
> > >> might
> > >> > be
> > >> > > a
> > >> > > > > > haste
> > >> > > > > > > > > decision.
> > >> > > > > > > > >
> > >> > > > > > > > > —
> > >> > > > > > > > > Denis
> > >> > > > > > > > >
> > >> > > > > > > > > > On Apr 5, 2017, at 7:42 AM,
Alexey Goncharuk <
> > >> > > > > > > > alexey.goncharuk@gmail.com>
> > >> > > > > > > > > wrote:
> > >> > > > > > > > > >
> > >> > > > > > > > > > +1, I see no other reasons
to keep it.
> > >> > > > > > > > > >
> > >> > > > > > > > > > 2017-04-05 13:59 GMT+03:00
Sergi Vladykin <
> > >> > > > > > sergi.vladykin@gmail.com
> > >> > > > > > > >:
> > >> > > > > > > > > >
> > >> > > > > > > > > >> +1
> > >> > > > > > > > > >>
> > >> > > > > > > > > >> Lets drop them.
> > >> > > > > > > > > >>
> > >> > > > > > > > > >> Sergi
> > >> > > > > > > > > >>
> > >> > > > > > > > > >> 2017-04-05 13:50 GMT+03:00
Dmitriy Govorukhin <
> > >> > > > > > > > > >> dmitriy.govorukhin@gmail.com>
> > >> > > > > > > > > >> :
> > >> > > > > > > > > >>
> > >> > > > > > > > > >>> Hi guys, i implemented
proxy for IgniteCache in
> > >> hibernate
> > >> > > > > > > > integration,
> > >> > > > > > > > > >> this
> > >> > > > > > > > > >>> proxy transformate
cacheKey to our key wrapper,
> > leaves
> > >> > only
> > >> > > > > > > required
> > >> > > > > > > > > >>> field. I think we
can remove identity resolve, it
> > >> should
> > >> > > not
> > >> > > > > > broke
> > >> > > > > > > > > >>> integration with hibernate.
Any objections?
> > >> > > > > > > > > >>>
> > >> > > > > > > > > >>> On Wed, Mar 29, 2017
at 11:07 PM, Valentin
> > Kulichenko
> > >> <
> > >> > > > > > > > > >>> valentin.kulichenko@gmail.com>
wrote:
> > >> > > > > > > > > >>>
> > >> > > > > > > > > >>>> I'm not saying
there is no alternative solution.
> > But
> > >> > let's
> > >> > > > > > > implement
> > >> > > > > > > > > it
> > >> > > > > > > > > >>> and
> > >> > > > > > > > > >>>> prove that it
works first, and remove resolvers
> > only
> > >> > after
> > >> > > > > that.
> > >> > > > > > > > > >>>>
> > >> > > > > > > > > >>>> -Val
> > >> > > > > > > > > >>>>
> > >> > > > > > > > > >>>> On Wed, Mar 29,
2017 at 12:18 PM, Sergi Vladykin
> <
> > >> > > > > > > > > >>> sergi.vladykin@gmail.com
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>> wrote:
> > >> > > > > > > > > >>>>
> > >> > > > > > > > > >>>>> Guys, nothing
is impossible if you know a bit
> > about
> > >> > > > > reflection
> > >> > > > > > in
> > >> > > > > > > > > >> Java
> > >> > > > > > > > > >>> :)
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>> We had a look
at the CacheKey class and it is
> > easily
> > >> > > > > > replaceable.
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>> Sergi
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>> 2017-03-29
21:49 GMT+03:00 Dmitriy Setrakyan <
> > >> > > > > > > > dsetrakyan@apache.org
> > >> > > > > > > > > >>> :
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>>> On Wed,
Mar 29, 2017 at 11:43 AM, Valentin
> > >> Kulichenko
> > >> > <
> > >> > > > > > > > > >>>>>> valentin.kulichenko@gmail.com>
wrote:
> > >> > > > > > > > > >>>>>>
> > >> > > > > > > > > >>>>>>> "Hibernate
key" is the CacheKey class I was
> > >> referring
> > >> > > to.
> > >> > > > > > It's
> > >> > > > > > > > > >>>> provided
> > >> > > > > > > > > >>>>>> by
> > >> > > > > > > > > >>>>>>> Hibernate,
not by user and not by us. So I'm
> not
> > >> sure
> > >> > > > it's
> > >> > > > > > > > > >> possible
> > >> > > > > > > > > >>>> to
> > >> > > > > > > > > >>>>>>> replace
it.
> > >> > > > > > > > > >>>>>>>
> > >> > > > > > > > > >>>>>>
> > >> > > > > > > > > >>>>>> If it
is impossible to replace or get rid of
> the
> > >> > > Hibernate
> > >> > > > > > key,
> > >> > > > > > > is
> > >> > > > > > > > > >>> this
> > >> > > > > > > > > >>>>>> discussion
valid at all?
> > >> > > > > > > > > >>>>>>
> > >> > > > > > > > > >>>>>
> > >> > > > > > > > > >>>>
> > >> > > > > > > > > >>>
> > >> > > > > > > > > >>
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message