ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Sapego <isap...@apache.org>
Subject Re: Stable binary key representation
Date Tue, 11 Apr 2017 18:46:17 GMT
Denis,

The whole binary representation of the object is used now
for hash code generation and equality comparison. So the
answer - all fields are used for this.

Best Regards,
Igor

On Mon, Apr 10, 2017 at 9:50 PM, Denis Magda <dmagda@apache.org> wrote:

> Considering this simple example
>
> INSERT (id, orgId, name, age, address) into Person…
>
> where id and orgId define Person’s affinity key - PersonKey(id, orgId)
>
> How do we know which fields to use for hash code generation and equality
> comparison? QueryEntity?
>
> No, it’s unclear how to document it properly.
>
> —
> Denis
>
> > On Apr 10, 2017, at 11:14 AM, Vladimir Ozerov <vozerov@gridgain.com>
> wrote:
> >
> > There is no more such resolver. It was removed.
> >
> > On Mon, Apr 10, 2017 at 8:58 PM, Denis Magda <dmagda@apache.org> wrote:
> >
> >> Vovan,
> >>
> >> Before I fix the documentation, what’t the replacement for
> >> BinaryFieldIdentiyResolver we used to define field for hash code
> >> calculation and equality comparison when DML statements are used?
> >> https://apacheignite.readme.io/docs/binary-marshaller#
> >> section-binary-field-identity-resolver <https://apacheignite.readme.
> >> io/docs/binary-marshaller#section-binary-field-identity-resolver>
> >>
> >> —
> >> Denis
> >>
> >>> On Apr 9, 2017, at 7:39 AM, Vladimir Ozerov <vozerov@gridgain.com>
> >> wrote:
> >>>
> >>> Resolvers were essential for DML because we had broken comparison
> >> semantics
> >>> of binary objects. This is not the case now.
> >>>
> >>> Resolver as a whole is normal practice. E.g. it is implemented in .NET
> on
> >>> core language level and widely used in many cases. Hazelcast has it as
> >> well
> >>> AFAIK. So it is wrong to think that the whole idea is useless. Think of
> >> it
> >>> as a comparator's brother.
> >>>
> >>> The only reason why we need to remove it is missing hash index in new
> >>> architecture. It makes sense, as it is better to have AI 2.0 without
> >> them,
> >>> than no AI 2.0 :-)
> >>>
> >>> 09 апр. 2017 г. 17:31 пользователь "Sergi Vladykin" <
> >>> sergi.vladykin@gmail.com> написал:
> >>>
> >>>> I guess Resolvers were added to DML just because they already existed
> >> since
> >>>> 1.9 and we were forced to support them in all the parts of our
> product.
> >>>>
> >>>> We have to stop this practice to add features without clear real life
> >> use
> >>>> cases for them.
> >>>>
> >>>> Sergi
> >>>>
> >>>> 2017-04-09 17:00 GMT+03:00 Denis Magda <dmagda@gridgain.com>:
> >>>>
> >>>>> Sergi, Vovan,
> >>>>>
> >>>>> Sorry for being annoying but I still didn't get an answer on whether
> >> the
> >>>>> resolvers are the must for DML. The main reason why we made them
up
> >> some
> >>>>> time ago is to support specific DML use cases. However I can't recall
> >> the
> >>>>> use cases.
> >>>>>
> >>>>> --
> >>>>> Denis
> >>>>>
> >>>>> On Sun, Apr 9, 2017 at 6:54 AM, Sergi Vladykin <
> >> sergi.vladykin@gmail.com
> >>>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Ok, we need to do 2 things here:
> >>>>>>
> >>>>>> 1. Drop the resolvers from the source code.
> >>>>>> 2. Write a good page in docs on "What makes a correct cache
key".
> >>>>>>
> >>>>>> Who can do that?
> >>>>>>
> >>>>>> Sergi
> >>>>>>
> >>>>>> 2017-04-07 9:48 GMT+03:00 Sergi Vladykin <sergi.vladykin@gmail.com
> >:
> >>>>>>
> >>>>>>> It is possible to try adding support of comparison to Resolvers,
> but
> >>>>> the
> >>>>>>> whole approach looks wrong and for now it is better to get
rid of
> it
> >>>>>> while
> >>>>>>> we have a chance to break compatibility.
> >>>>>>>
> >>>>>>> Sergi
> >>>>>>>
> >>>>>>> 2017-04-07 9:19 GMT+03:00 Valentin Kulichenko <
> >>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>
> >>>>>>>> The discussion should've been started with that :) If
supporting
> >>>>>> resolvers
> >>>>>>>> in new architecture is not possible or means too big
effort, then
> >>>> it's
> >>>>>>>> definitely not worth it.
> >>>>>>>>
> >>>>>>>> -Val
> >>>>>>>>
> >>>>>>>> On Thu, Apr 6, 2017 at 8:52 PM, Vladimir Ozerov <
> >>>> vozerov@gridgain.com
> >>>>>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Dima,
> >>>>>>>>>
> >>>>>>>>> Yes, they may explode some internals of our indexes.
> >>>>>>>>>
> >>>>>>>>> 06 апр. 2017 г. 23:32 пользователь
"Dmitriy Setrakyan" <
> >>>>>>>>> dsetrakyan@apache.org> написал:
> >>>>>>>>>
> >>>>>>>>>> Guys,
> >>>>>>>>>>
> >>>>>>>>>> Isn't the main issue here that we cannot use
the Identity
> >>>>> Resolvers
> >>>>>> in
> >>>>>>>>>> BTrees in the 2.0 version? If yes, then we have
to remove them
> >>>> no
> >>>>>>>> matter
> >>>>>>>>>> what.
> >>>>>>>>>>
> >>>>>>>>>> D.
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Apr 6, 2017 at 1:23 PM, Sergi Vladykin
<
> >>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Binary key representation is stable when
we always have equal
> >>>>>>>>> serialized
> >>>>>>>>>>> bytes when the original keys are equal.
> >>>>>>>>>>>
> >>>>>>>>>>> Resolver allows you to have some extra info
in the Key and
> >>>> equal
> >>>>>>>> Keys
> >>>>>>>>>> will
> >>>>>>>>>>> be serialized into different bytes, which
is wrong.
> >>>>>>>>>>>
> >>>>>>>>>>> Look at the example what you can do with
resolvers:
> >>>>>>>>>>>
> >>>>>>>>>>> We may have some data entry with fields
a, b, c. Let's say the
> >>>>>>>> unique
> >>>>>>>>>> part
> >>>>>>>>>>> here is `a` and it the only fields used
in Key equals() and
> >>>>>>>> hashCode().
> >>>>>>>>>>> Still we may have the following layouts:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. Ka -> Vbc
> >>>>>>>>>>> 2. Kab -> Vc
> >>>>>>>>>>> 3. Kabc -> Boolean.TRUE
> >>>>>>>>>>>
> >>>>>>>>>>> The only 1 is a correct layout, others are
plain wrong
> >>>> variants
> >>>>>> (but
> >>>>>>>>> they
> >>>>>>>>>>> are still possible with Resolvers) because
everything that
> >>>> does
> >>>>>> not
> >>>>>>>>> make
> >>>>>>>>>>> Key unique must be in Value.
> >>>>>>>>>>>
> >>>>>>>>>>> We want to clearly state that if you have
something in Key,
> >>>> that
> >>>>>> is
> >>>>>>>> not
> >>>>>>>>>>> part of equals(), then the Key is invalid
and that stuff must
> >>>> be
> >>>>>> in
> >>>>>>>>>> Value.
> >>>>>>>>>>> This allows us to rely on binary representation
of a Key to be
> >>>>>>>> stable
> >>>>>>>>> and
> >>>>>>>>>>> have some more optimizations and code simplifications
with
> >>>>> respect
> >>>>>>>> to
> >>>>>>>>>> these
> >>>>>>>>>>> assumptions.
> >>>>>>>>>>>
> >>>>>>>>>>> Sergi
> >>>>>>>>>>>
> >>>>>>>>>>> 2017-04-06 14:24 GMT+03:00 Valentin Kulichenko
<
> >>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>
> >>>>>>>>>>>> Even with my vast expirience I would
never claim that I've
> >>>>> seen
> >>>>>>>>>>>> "everything" :)
> >>>>>>>>>>>>
> >>>>>>>>>>>> What do you mean by stable binary key
representation and how
> >>>>>>>>> resolvers
> >>>>>>>>>>> make
> >>>>>>>>>>>> it unstable?
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Val
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Apr 6, 2017 at 2:36 AM, Sergi
Vladykin <
> >>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Val,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I know that you have really vast
experience in Ignite
> >>>>>>>> deployments
> >>>>>>>>> and
> >>>>>>>>>>>>> probably saw everything that can
happen. Did you ever see
> >>>>>>>> identity
> >>>>>>>>>>>>> resolvers use in real life? I guess
no.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hibernate example is bad here, because
if their key is
> >>>>>> unstable
> >>>>>>>>>> across
> >>>>>>>>>>>>> multiple JVMs, it means that it
was not designed for
> >>>>>> distributed
> >>>>>>>>>>> caches a
> >>>>>>>>>>>>> priori.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also knowing in advance about stable
binary key
> >>>>> representation
> >>>>>>>>> allows
> >>>>>>>>>>> us
> >>>>>>>>>>>> to
> >>>>>>>>>>>>> apply additional optimizations,
like comparing keys
> >>>> without
> >>>>>>>>> detaching
> >>>>>>>>>>>> them
> >>>>>>>>>>>>> from offheap memory.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We always will be able to add this
stuff back if we see
> >>>>> users
> >>>>>>>>> really
> >>>>>>>>>>> need
> >>>>>>>>>>>>> it. Let's remove it for 2.0.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2017-04-06 11:21 GMT+03:00 Valentin
Kulichenko <
> >>>>>>>>>>>>> valentin.kulichenko@gmail.com>:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Alex,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> To be honest, I don't understand
the reasoning behind
> >>>> the
> >>>>>>>>> removal.
> >>>>>>>>>> I
> >>>>>>>>>>>>> think
> >>>>>>>>>>>>>> resolvers provide good flexibility
for different corner
> >>>>>> cases
> >>>>>>>> and
> >>>>>>>>>>> it's
> >>>>>>>>>>>> a
> >>>>>>>>>>>>>> good thing to have them. Note
that they can be applied
> >>>> not
> >>>>>>>> only
> >>>>>>>>> to
> >>>>>>>>>>>> cache
> >>>>>>>>>>>>>> keys, but to any binary objects.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hibernate issue is actually
a good example of such use
> >>>>> case.
> >>>>>>>> The
> >>>>>>>>>> fact
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>> we found an alternative solution
doesn't actually mean
> >>>>>>>> anything,
> >>>>>>>>>>>> because
> >>>>>>>>>>>>>> what if this happened not in
our module, but in user's
> >>>>>>>>> application?
> >>>>>>>>>>>>>> Unfortunately, we can't predict
everything.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Error proneness is not a very
strong argument either,
> >>>>>> because
> >>>>>>>> in
> >>>>>>>>> my
> >>>>>>>>>>>> view
> >>>>>>>>>>>>>> these resolvers are as much
error prone as
> >>>> BinaryIdMapper,
> >>>>>> for
> >>>>>>>>>>> example.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 11:44
PM, Alexey Goncharuk <
> >>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Denis,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Can you suggest a use-case
where identity resolver is
> >>>>>> needed
> >>>>>>>>>> (given
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>> agree that a key must contain
only valuable fields)?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2017-04-05 22:08 GMT+03:00
Denis Magda <
> >>>>> dmagda@apache.org
> >>>>>>> :
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Where do you want to
remove the identity resolvers
> >>>>> from?
> >>>>>>>> If
> >>>>>>>>>> it’s
> >>>>>>>>>>>>>> related
> >>>>>>>>>>>>>>>> to the internals of
Hibernate module then it’s fine
> >>>>> but
> >>>>>> if
> >>>>>>>>> you
> >>>>>>>>>>>>> suggest
> >>>>>>>>>>>>>>>> removing identity resolvers
public interfaces then
> >>>> it
> >>>>>>>> might
> >>>>>>>>> be
> >>>>>>>>>> a
> >>>>>>>>>>>>> haste
> >>>>>>>>>>>>>>>> decision.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Apr 5, 2017,
at 7:42 AM, Alexey Goncharuk <
> >>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> +1, I see no other
reasons to keep it.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2017-04-05 13:59
GMT+03:00 Sergi Vladykin <
> >>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> +1
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Lets drop them.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Sergi
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 2017-04-05 13:50
GMT+03:00 Dmitriy Govorukhin <
> >>>>>>>>>>>>>>>>>> dmitriy.govorukhin@gmail.com>
> >>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi guys,
i implemented proxy for IgniteCache in
> >>>>>>>> hibernate
> >>>>>>>>>>>>>>> integration,
> >>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>> proxy transformate
cacheKey to our key wrapper,
> >>>>>> leaves
> >>>>>>>>> only
> >>>>>>>>>>>>>> required
> >>>>>>>>>>>>>>>>>>> field. I
think we can remove identity resolve,
> >>>> it
> >>>>>>>> should
> >>>>>>>>>> not
> >>>>>>>>>>>>> broke
> >>>>>>>>>>>>>>>>>>> integration
with hibernate. Any objections?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed,
Mar 29, 2017 at 11:07 PM, Valentin
> >>>>>> Kulichenko
> >>>>>>>> <
> >>>>>>>>>>>>>>>>>>> valentin.kulichenko@gmail.com>
wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I'm
not saying there is no alternative
> >>>> solution.
> >>>>>> But
> >>>>>>>>> let's
> >>>>>>>>>>>>>> implement
> >>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>> prove
that it works first, and remove resolvers
> >>>>>> only
> >>>>>>>>> after
> >>>>>>>>>>>> that.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> -Val
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed,
Mar 29, 2017 at 12:18 PM, Sergi
> >>>> Vladykin
> >>>>> <
> >>>>>>>>>>>>>>>>>>> sergi.vladykin@gmail.com
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
Guys, nothing is impossible if you know a bit
> >>>>>> about
> >>>>>>>>>>>> reflection
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> Java
> >>>>>>>>>>>>>>>>>>> :)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
We had a look at the CacheKey class and it is
> >>>>>> easily
> >>>>>>>>>>>>> replaceable.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
Sergi
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
2017-03-29 21:49 GMT+03:00 Dmitriy Setrakyan <
> >>>>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
On Wed, Mar 29, 2017 at 11:43 AM, Valentin
> >>>>>>>> Kulichenko
> >>>>>>>>> <
> >>>>>>>>>>>>>>>>>>>>>>
valentin.kulichenko@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
"Hibernate key" is the CacheKey class I was
> >>>>>>>> referring
> >>>>>>>>>> to.
> >>>>>>>>>>>>> It's
> >>>>>>>>>>>>>>>>>>>> provided
> >>>>>>>>>>>>>>>>>>>>>>
by
> >>>>>>>>>>>>>>>>>>>>>>>
Hibernate, not by user and not by us. So I'm
> >>>>> not
> >>>>>>>> sure
> >>>>>>>>>>> it's
> >>>>>>>>>>>>>>>>>> possible
> >>>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>>>>
replace it.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
If it is impossible to replace or get rid of
> >>>>> the
> >>>>>>>>>> Hibernate
> >>>>>>>>>>>>> key,
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>>>>>>
discussion valid at all?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message