ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@gridgain.com>
Subject Re: Missing ways to get access to Ignite CacheEntry
Date Mon, 21 Dec 2015 08:11:34 GMT
Why it shouldn't work?
If client receives an entry that has a reference to the entry with 
version, transferred from a server as well as a part of the response, 
then unwrap(...) should work.
Isn't this feasible to implement?

--
Denis

On 12/21/2015 10:59 AM, Dmitriy Setrakyan wrote:
> Again, my issue is with unwrap method. Are we all in agreement that
> unwrap(…) will not work on the client side?
>
> D.
>
> On Sun, Dec 20, 2015 at 11:30 PM, Denis Magda <dmagda@gridgain.com> wrote:
>
>> Actually I think that Andrey's proposal on addition of
>> IgniteCache.getEntry(k) sounds natural.
>>
>> Is there any platform limitation that will prevent us from loading an
>> entry with its version from a remote node when such a particular method is
>> used?
>> I think that it's feasible to support it and in addition I would add on
>> more method to the API - IgniteCache.getAllEntries(keysSet).
>>
>> --
>> Denis
>>
>>
>> On 12/19/2015 2:05 AM, Dmitriy Setrakyan wrote:
>>
>>> On Fri, Dec 18, 2015 at 1:23 PM, Andrey Kornev <andrewkornev@hotmail.com>
>>> wrote:
>>>
>>> How would I know how to interpret the returned null?
>>>> Can we just have a simple getEntry(K) method that has semantics of the
>>>> regular get(K)? Why invent something?
>>>>
>>>> If we do this, how would you unwrap such entry on the client side?
>>>
>>> Cheers
>>>> Andrey
>>>>
>>>> From: dsetrakyan@apache.org
>>>>> Date: Fri, 18 Dec 2015 12:55:47 -0800
>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry
>>>>> To: dev@ignite.apache.org
>>>>>
>>>>> On Fri, Dec 18, 2015 at 12:40 PM, Andrey Kornev <
>>>>>
>>>> andrewkornev@hotmail.com>
>>>>
>>>>> wrote:
>>>>>
>>>>> Dmitriy,
>>>>>> I understand that we're talking about Ignite CacheEntry and not
>>>>>>
>>>>> JCache's
>>>>> CacheEntry that has no version property to speak of.
>>>>>> But I don't understand why the method is named localEntry()? What
does
>>>>>> "local" mean in this context? Does it mean that I can only use it
on
>>>>>>
>>>>> the
>>>>> primary node for a given key? What would happen if I call this method
>>>>> on a
>>>>> non-primary node? And what would I do if I needed the entry (along
>>>>> with the
>>>>> valid version it contains) on a non-primary node?
>>>>>> If the node is either primary or backup, then you will get the entry
>>>> back.
>>>>
>>>>> Otherwise, you will get null.
>>>>>
>>>>>
>>>>> From: dsetrakyan@apache.org
>>>>>>> Date: Fri, 18 Dec 2015 12:19:01 -0800
>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry
>>>>>>> To: dev@ignite.apache.org
>>>>>>>
>>>>>>> On Fri, Dec 18, 2015 at 9:47 AM, Andrey Kornev <
>>>>>>>
>>>>>> andrewkornev@hotmail.com
>>>>> wrote:
>>>>>>> Dmitriy,
>>>>>>>> Based on what criteria does one determine which information
is
>>>>>>>>
>>>>>>> redundant
>>>>>>> and which is not?
>>>>>>>> To me, if an API declares a method getSomeRedundantInfo(),
then the
>>>>>>>> implementer has no choice, but to comply with the contract
and
>>>>>>>>
>>>>>>> provide
>>>>> such
>>>>>>> info. One can't just return a null or ignore it some other way,
>>>>>>> simply
>>>>> because it appears to have some performance impact.
>>>>>>> Andrey, the CacheEntry is not the JCache entry, but rather the
>>>>>>>
>>>>>> internal
>>>>> Ignite entry that you can get when unwrapping the JCache entry. The
>>>>>>> unwrapping, obviously, can only happen on the server side.
>>>>>>>
>>>>>> Therefore, I
>>>>> am
>>>>>>> suggesting that we also add a localEntry(K) method which will
>>>>>>>
>>>>>> provide the
>>>>> entry API on the server side directly (not from iterator or EP).
>>>>>>> Now I understand that the version() method currently does not
work
>>>>>>>
>>>>>> properly
>>>>>>
>>>>>>> on the server side, but this is something we can easily fix.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>>
>>>>>>> Besides, the overhead of the version number may turn out to be
>>>>>>> negligible
>>>>>>> when benchmarked. Most likely there will be near zero overhead
>>>>>>> (due to
>>>>> the
>>>>>>> way the network stack packs the bytes into the ethernet frames).
>>>>>>> The
>>>>> workaround for the broken API however may be much more expensive in
>>>>>>> terms
>>>>>>> of performance and user experience.
>>>>>>>> Please let's keep the API consistent and predictable.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Andrey
>>>>>>>>
>>>>>>>> From: dsetrakyan@apache.org
>>>>>>>>> Date: Fri, 18 Dec 2015 08:41:00 -0800
>>>>>>>>> Subject: Re: Missing ways to get access to Ignite CacheEntry
>>>>>>>>> To: dev@ignite.apache.org
>>>>>>>>>
>>>>>>>>> Andrey,
>>>>>>>>>
>>>>>>>>> The main issue with returning version is sending redundant
>>>>>>>>>
>>>>>>>> information
>>>>>>> over
>>>>>>>>> the wire. What if we added a method IgniteCache.localEntry(K)
>>>>>>>>>
>>>>>>>> which
>>>>> would
>>>>>>> return a local entry instance with a version. You would only
be
>>>>>>>> able
>>>>> to
>>>>>>> invoke this method on the server node (not client).
>>>>>>>>> Would this work for you?
>>>>>>>>>
>>>>>>>>> D.
>>>>>>>>>
>>>>>>>>> On Fri, Dec 18, 2015 at 8:23 AM, Andrey Kornev <
>>>>>>>>>
>>>>>>>> andrewkornev@hotmail.com
>>>>>>> wrote:
>>>>>>>>> Denis,
>>>>>>>>>> I'm not suggesting to have every cache read operation
return a
>>>>>>>>>>
>>>>>>>>> CacheEntry.
>>>>>>>>> The "classical" GridGain used to have a getEntry(key)
method
>>>>>>>>> that
>>>>> was
>>>>>>> very
>>>>>>>>> useful. The current version of Ignite provides virtually
no
>>>>>>>>> way of
>>>>> obtaining a cache entry except thru an iterator, or using an
>>>>>>>>> EntryProcessor
>>>>>>>>> invocation which seems more like a hack.
>>>>>>>>>> As for the question of what's stored in Ignite cache
and the
>>>>>>>>>>
>>>>>>>>> pre-compute,
>>>>>>>>> it is really an off-topic. It just came about as we were
>>>>>>>>> looking
>>>>> for a
>>>>>>> way
>>>>>>>>> to work around the issue with Ignite query not returning
cache
>>>>>>>>> entries'
>>>>>>> version numbers in the result iterator. (There is now a ticket
>>>>>>>>> that
>>>>> addresses this issue:
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-2202
>>>>>>>>> )This is when we realized that there is no longer any
way in
>>>>>>>>> Ignite to
>>>>>>> retrieve the CacheEntry by its key.
>>>>>>>>>> But for the sake of completeness I quickly go over
the entire
>>>>>>>>>>
>>>>>>>>> use
>>>>> case,
>>>>>>> because it seems I wasn't clear enough in the previous post.
>>>>>>>>>> Ignite cache holds the data entries that are used
as the input
>>>>>>>>>>
>>>>>>>>> into
>>>>> some
>>>>>>>>> expensive computation. The data in the cache gets updated
from
>>>>>>>>> time to
>>>>>>> time, the computation only needs to be re-run
>>>>>>>>>> - on-demand -- when a user asks for the computed
value and
>>>>>>>>>> - the pre-computed value is obsolete -- the source
cache data
>>>>>>>>>>
>>>>>>>>> has
>>>>> been
>>>>>>> updated since the last time the value was computed.
>>>>>>>>>> Otherwise, we return the pre-computed value. Note
that we don't
>>>>>>>>>>
>>>>>>>>> want to
>>>>>>> rerun the computation for each cache update! Only when the
>>>>>>>>> users
>>>>> asks
>>>>>>> for
>>>>>>>>> it.
>>>>>>>>>> I hope it make it more clear now.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> Andrey
>>>>>>>>>>
>>>>>>>>>> Subject: Re: Missing ways to get access to Ignite
CacheEntry
>>>>>>>>>>> To: dev@ignite.apache.org
>>>>>>>>>>> From: dmagda@gridgain.com
>>>>>>>>>>> Date: Fri, 18 Dec 2015 13:44:16 +0300
>>>>>>>>>>>
>>>>>>>>>>> Hi Andrey,
>>>>>>>>>>>
>>>>>>>>>>> I got you, thanks for the clarification.
>>>>>>>>>>>
>>>>>>>>>>> So since you're storing a computed value in some
local data
>>>>>>>>>>>
>>>>>>>>>> structure
>>>>>>> what is stored in the Ignite cache as a value in such a case?
>>>>>>>>>> There
>>>>>>> should be something.
>>>>>>>>>>> Why don't you (or can't you) store a version
identifier in
>>>>>>>>>>>
>>>>>>>>>> that
>>>>> value
>>>>>>> that is located in the Ignite cache? This should work
>>>>>>>>>> perfectly
>>>>> well
>>>>>>> for
>>>>>>>>> you use-case.
>>>>>>>>>>> Personally I'm against adding CacheEntry to a
response of
>>>>>>>>>>>
>>>>>>>>>> every
>>>>> get
>>>>>>> like
>>>>>>>>> operation that happens over a cache (get, getAll, SQL
& Scan
>>>>>>>>>> queries).
>>>>>>>>> This will add extra size to every response and only in
rare
>>>>>>>>>> cases
>>>>> someone will benefit from it.
>>>>>>>>>>> However, what if we add a special configuration
parameter to
>>>>>>>>>>> CacheConfiguration that will manage whether CacheEntry
has
>>>>>>>>>>>
>>>>>>>>>> to be
>>>>> passed
>>>>>>>>> as a part of a response or not? Igniters, what do you
think
>>>>>>>>>> on
>>>>> this?
>>>>>>>>>>> Regards,
>>>>>>>>>>> Denis
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 12/17/2015 9:09 AM, Andrey Kornev wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Denis,In this case I used the term "caching"
in general
>>>>>>>>>>>>
>>>>>>>>>>> sense,
>>>>> meaning
>>>>>>>>> saving the computed value for later. I didn't mean the
Ignite
>>>>>>>>> cache.Sorry
>>>>>>>>> about the confusion.Andrey
>>>>>>>>>>>        _____________________________
>>>>>>>>>>>> From: Denis Magda <dmagda@gridgain.com>
>>>>>>>>>>>> Sent: Wednesday, December 16, 2015 9:48 PM
>>>>>>>>>>>> Subject: Re: Missing ways to get access to
Ignite
>>>>>>>>>>>>
>>>>>>>>>>> CacheEntry
>>>>> To:  <dev@ignite.apache.org>
>>>>>>>>>>>>
>>>>>>>>>>>>                       Hi Romain,
>>>>>>>>>>>>
>>>>>>>>>>>>     I'm a bit confused cause initially you
saying that
>>>>>>>>>>>>
>>>>>>>>>>>>     /The value is computed on first access
and it is then
>>>>>>>>>>>>
>>>>>>>>>>> cached
>>>>>     /and then you add that
>>>>>>>>>>>>     /One constraint is that the computed
value is not
>>>>>>>>>>>>
>>>>>>>>>>> serializable.
>>>>>>> //
>>>>>>>>>     /
>>>>>>>>>>>>
>>>>>>>>>>>>     Ignite won't be able to store a value
of cache if
>>>>>>>>>>>>
>>>>>>>>>>> serialization
>>>>>>> is
>>>>>>>>> not
>>>>>>>>>>>     supported for it by some reason.
>>>>>>>>>>>>     Please clarify here. Probably I'm missing
something from
>>>>>>>>>>>>
>>>>>>>>>>> your
>>>>> description.
>>>>>>>>>>>     --
>>>>>>>>>>>>     Denis
>>>>>>>>>>>>
>>>>>>>>>>>>     On 12/16/2015 6:21 PM, Andrey Kornev
wrote:
>>>>>>>>>>>>     > Romain
>>>>>>>>>>>>     >
>>>>>>>>>>>>     > I suggest you be very careful using
the invoke()
>>>>>>>>>>>>
>>>>>>>>>>> functionality. As
>>>>>>>>> explained in this posting (and the follow-ups)
>>>>>>>>>>>     >
>>>> http://mail-archives.apache.org/mod_mbox/ignite-dev/201511.mbox/%3CSNT147-W14D2736E1374450B685721CF000%40phx.gbl%3E
>>>>
>>>>>     > the correct usage of the invoke() may be difficult, to
>>>>>>>>>>> put
>>>>> it
>>>>>>> politely.
>>>>>>>>>>>     >
>>>>>>>>>>>>     > @Denis Very briefly, the use case
is "conditional
>>>>>>>>>>>>
>>>>>>>>>>> re-compute".
>>>>>>> There is a pretty expensive computation that needs to be
>>>>>>>>> applied to
>>>>> each
>>>>>>>>> entry on demand. The value is computed on first access
and it
>>>>>>>>> is
>>>>> then
>>>>>>> cached along with the current version of the cache entry.  The
>>>>>>>>> value
>>>>>>> must
>>>>>>>>> be recomputed on next access only if the entry has been
>>>>>>>>> modified.
>>>>>     >
>>>>>>>>>>>>     > If anyone has a better idea how
to achieve something
>>>>>>>>>>>>
>>>>>>>>>>> like
>>>>> that,
>>>>>>> please do share! One constraint is that the computed value is
>>>>>>>>> not
>>>>> serializable.
>>>>>>>>>>>     >
>>>>>>>>>>>>     > Cheers
>>>>>>>>>>>>     > Andrey
>>>>>>>>>>>>     >
>>>>>>>>>>>>     >> Subject: Re: Missing ways to
get access to Ignite
>>>>>>>>>>>>
>>>>>>>>>>> CacheEntry
>>>>>>>     >> To: dev@ignite.apache.org
>>>>>>>>>>>>     >> From: dmagda@gridgain.com
>>>>>>>>>>>>     >> Date: Wed, 16 Dec 2015 16:11:48
+0300
>>>>>>>>>>>>     >>
>>>>>>>>>>>>     >> Romain,
>>>>>>>>>>>>     >>
>>>>>>>>>>>>     >> If it's implemented this way
for now it doesn't mean
>>>>>>>>>>>>
>>>>>>>>>>> that
>>>>> we
>>>>>>> can't
>>>>>>>>>     >> enhance the API for other use cases ;)
>>>>>>>>>>>>     >> As I said presently it's supported
for limited number
>>>>>>>>>>>>
>>>>>>>>>>> of
>>>>> methods
>>>>>>>>> mostly
>>>>>>>>>>>     >> by performance reasons.
>>>>>>>>>>>>     >>
>>>>>>>>>>>>     >> Please go ahead and try invoke/invokeAll
and if it
>>>>>>>>>>>>
>>>>>>>>>>> doesn't
>>>>> work
>>>>>>>>> fine for
>>>>>>>>>>>     >> you we can keep discussing what
to do next.
>>>>>>>>>>>>     >>
>>>>>>>>>>>>     >> Regards,
>>>>>>>>>>>>     >> Denis
>>>>>>>>>>>>     >>
>>>>>>>>>>>>     >> On 12/16/2015 1:53 PM, Romain
Gilles wrote:
>>>>>>>>>>>>     >>> Hi Denis,
>>>>>>>>>>>>     >>> Thanks for you replay. And
sorry to not double check
>>>>>>>>>>>>
>>>>>>>>>>> it
>>>>> before.
>>>>>>>>> I see that
>>>>>>>>>>>     >>> if I want to work with CacheEntry,
I need to use the
>>>>>>>>>>> invoke
>>>>>>> method in order
>>>>>>>>>>>     >>> to return the CacheEntry. This
is the way I should do
>>>>>>>>>>> it. It
>>>>>>> sounds like
>>>>>>>>>>>     >>> complicated for "just" getting
an entry. But if you
>>>>>>>>>>> say
>>>>> this
>>>>>>> is
>>>>>>>>> the way I
>>>>>>>>>>>     >>> will do it like that. I was
just think that it could
>>>>>>>>>>> be a
>>>>> common
>>>>>>>>> use case
>>>>>>>>>>>     >>> and therefore provide it as
a shortcut.
>>>>>>>>>>>>     >>>
>>>>>>>>>>>>     >>> Thanks,
>>>>>>>>>>>>     >>>
>>>>>>>>>>>>     >>> Romain.
>>>>>>>>>>>>     >>>
>>>>>>>>>>>>     >>> Le mer. 16 déc. 2015 à
11:34, Denis Magda <
>>>>>>>>>>>>
>>>>>>>>>>> dmagda@gridgain.com>
>>>>>>>>> a écrit :
>>>>>>>>>>>     >>>
>>>>>>>>>>>>     >>>> Hi Romain,
>>>>>>>>>>>>     >>>>
>>>>>>>>>>>>     >>>> As the current documentation
of
>>>>>>>>>>>>
>>>>>>>>>>> org.apache.ignite.cache.CacheEntry
>>>>>>>>>>>     >>>> states it's possible to
get a reference to
>>>>>>>>>>> CacheEntry
>>>>> and
>>>>>>> its
>>>>>>>>> version
>>>>>>>>>>>     >>>> only for methods that are
executed over local data
>>>>>>>>>>> set
>>>>> stored
>>>>>>>>> on a node.
>>>>>>>>>>>     >>>> Among such methods are invoke
& invokeAll and
>>>>>>>>>>> randomEntry.
>>>>>>>     >>>>
>>>>>>>>>>>>     >>>> You probably can get
a CacheEntry and its non null
>>>>>>>>>>>>
>>>>>>>>>>> version
>>>>>>> when
>>>>>>>>> a cache
>>>>>>>>>>>     >>>> iterator is in use but the
version will be 'null',
>>>>>>>>>>> as
>>>>> far
>>>>>>> as I
>>>>>>>>> remember,
>>>>>>>>>>>     >>>> for those entries that are
loaded from remote nodes.
>>>>>>>>>>>>     >>>> Presently Ignite doesn't
transfer the version from
>>>>>>>>>>>>
>>>>>>>>>>> remote
>>>>>>> nodes
>>>>>>>>> as a
>>>>>>>>>>>     >>>> part of response by perform
reasons.
>>>>>>>>>>>>     >>>>
>>>>>>>>>>>>     >>>> Please elaborate more
on your particular use case
>>>>>>>>>>>>
>>>>>>>>>>> and
>>>>> what
>>>>>>> API
>>>>>>>>> you would
>>>>>>>>>>>     >>>> like to add in order to
support it.
>>>>>>>>>>>>     >>>>
>>>>>>>>>>>>     >>>> --
>>>>>>>>>>>>     >>>> Denis
>>>>>>>>>>>>     >>>>
>>>>>>>>>>>>     >>>> On 12/16/2015 12:58
PM, Romain Gilles wrote:
>>>>>>>>>>>>     >>>>> Hi Igniters,
>>>>>>>>>>>>     >>>>> I'm looking for
a way to get access to the Ignite
>>>>>>>>>>>>
>>>>>>>>>>> CacheEntry.
>>>>>>>>> For now
>>>>>>>>>>>     >>>> this
>>>>>>>>>>>>     >>>>> is the ways I found:
>>>>>>>>>>>>     >>>>>
>>>>>>>>>>>>     >>>>>       - Through
the queris
>>>>>>>>>>>>     >>>>>       - Through
jsr 107 Cache Iterable
>>>>>>>>>>>>     >>>>>       - Through
jsr 107 Cache itterator
>>>>>>>>>>>>     >>>>>       - Through
IgniteCache::randomEntry()
>>>>>>>>>>>>     >>>>>
>>>>>>>>>>>>     >>>>> If I remember correctly
it was possible to get the
>>>>>>>>>>>>
>>>>>>>>>>> CacheEntry
>>>>>>>>> from a
>>>>>>>>>>>     >>>> given
>>>>>>>>>>>>     >>>>> key in old version
of gridgain community
>>>>>>>>>>>>     >>>>> version: GridCacheProjection::entry(K
key)
>>>>>>>>>>>>
>>>>>>>>>>> GridCacheEntry<K,V>
>>>>>>>>>     >>>>> I think it could be a good to
introduce this
>>>>>>>>>>> feature at
>>>>> IgniteCache
>>>>>>>>>>>     >>>> level.
>>>>>>>>>>>>     >>>>> Or maybe there is
an other way to do it.
>>>>>>>>>>>>     >>>>>
>>>>>>>>>>>>     >>>>> Thanks,
>>>>>>>>>>>>     >>>>>
>>>>>>>>>>>>     >>>>> Romain.
>>>>>>>>>>>>     >>>>>
>>>>>>>>>>>>     >
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>


Mime
View raw message