ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@gridgain.com>
Subject Re: IgniteCache.invoke on ALL keys
Date Tue, 31 May 2016 12:44:47 GMT
Pavel,

> This seems to be a common task, why don't we implement invokeAll as I
> suggested above?

I would implement such a method using the approach shown in the example. In my understanding
it would be the most efficient way. Data locality will longer be not an issue when IGNITE-2310
is implemented.

—
Denis

> On May 31, 2016, at 1:16 PM, Pavel Tupitsyn <ptupitsyn@gridgain.com> wrote:
> 
> Denis, thank you, this may work, but:
> * it is too complicated
> * it does not guarantee locality
> 
> This seems to be a common task, why don't we implement invokeAll as I
> suggested above?
> 
> On Tue, May 31, 2016 at 1:05 PM, Denis Magda <dmagda@gridgain.com> wrote:
> 
>> Pavel,
>> 
>> Here is an example on how to execute scan queries on partitions’ owners
>> and perform some operation for every entry
>> 
>> https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/datagrid/query/ScanQueryExample.java
>> <
>> https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/datagrid/query/ScanQueryExample.java
>>> 
>> 
>> When this feature is implemented [1] it will be possible to postpone
>> partitions movement until such an operation is in progress. Presently if
>> the rebalancing happens the data will be retrieved from a remote node (new
>> partition owner), however the result will be consistent in any case.
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-2310 <
>> https://issues.apache.org/jira/browse/IGNITE-2310>
>> 
>> —
>> Denis
>> 
>>> On May 31, 2016, at 7:42 AM, Yakov Zhdanov <yzhdanov@gridgain.com>
>> wrote:
>>> 
>>> Vova, even though it operates on single key we will need to "pin"
>> partition
>>> so key does not go to another node.
>>> 
>>> Pavel, you can also send closures to all primary nodes to do local scan
>>> query for each partition. This way you will go over each entry.
>>> 
>>> Thanks!
>>> --
>>> Yakov Zhdanov, Director R&D
>>> *GridGain Systems*
>>> www.gridgain.com
>>> 
>>> 2016-05-30 16:05 GMT-04:00 Vladimir Ozerov <vozerov@gridgain.com>:
>>> 
>>>> Affinity run/call operate on a single key AFAIK.
>>>> 
>>>> On Mon, May 30, 2016 at 10:55 PM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org
>>>>> 
>>>> wrote:
>>>> 
>>>>> Actually I have seen a ticket to block moving partitions if
>> affinityCall
>>>> or
>>>>> affinityRun are called. I think once these tickets are implemented, the
>>>>> process will become reliable, no?
>>>>> 
>>>>> D.
>>>>> 
>>>>> On Mon, May 30, 2016 at 9:13 AM, Pavel Tupitsyn <
>> ptupitsyn@gridgain.com>
>>>>> wrote:
>>>>> 
>>>>>> Dmitriy, as I understand, there is no reliable way to do that if
>>>>>> rebalancing happens.
>>>>>> 
>>>>>> On Mon, May 30, 2016 at 6:50 PM, Dmitriy Setrakyan <
>>>>> dsetrakyan@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>>> I think we do support this use case. Why not send a computation
to a
>>>>>> server
>>>>>>> and then perform the iteration through the cache entries locally
on
>>>>> that
>>>>>>> server?
>>>>>>> 
>>>>>>> On Mon, May 30, 2016 at 4:44 AM, Pavel Tupitsyn <
>>>>> ptupitsyn@gridgain.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Igniters,
>>>>>>>> 
>>>>>>>> Looks like we do not have an efficient way to perform an
action on
>>>>>> EVERY
>>>>>>>> cache entry.
>>>>>>>> 
>>>>>>>> Let's say I want to remove all entries that match a predicate.
>>>>>>>> My only option is to retrieve these entries via Scan or SQL
query,
>>>>> and
>>>>>>> then
>>>>>>>> call removeAll.
>>>>>>>> This involves a lot of unnecessary network trips (send keys
to
>>>> caller
>>>>>>> node,
>>>>>>>> send them back to primary nodes).
>>>>>>>> 
>>>>>>>> Would it be possible to implement a method like
>>>>>>>> void IgniteCache.invokeAll(entryProcessor)
>>>>>>>> that invokes the processor on all entries and does not return
>>>>> anything?
>>>>>>>> There could be more overloads that return results or only
return
>>>>>> results
>>>>>>>> for changed entries.
>>>>>>>> 
>>>>>>>> Thoughts?
>>>>>>>> 
>>>>>>>> Pavel.
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> 
>> 


Mime
View raw message