ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yakov Zhdanov <yzhda...@apache.org>
Subject Re: DataStreamer performance with removing data
Date Wed, 11 Nov 2015 10:44:36 GMT
It seems to be better to broadcast closure, query local keys and call
clear() for them locally. If remove semantics needed then closure should
query primary keys, then remove should be called.

--Yakov

2015-11-11 2:29 GMT+03:00 vkulichenko <valentin.kulichenko@gmail.com>:

> Hi Kevin,
>
> The difference is that data streamer uses remove() method to update the
> cache, while your second test uses clearAll(). remove() and removeAll() are
> transactional operations and it also updates persistence store if you have
> it. clearAll(), on the other side, only removes the data from the in-memory
> cache. It will also skip entries that are currently enlisted in
> transactions
> or locked. To make this comparison fair, you should use removeAll() instead
> of clearAll(). Can you try it?
>
> If you're OK with semantics of clearAll() method, you can just use it. You
> also can create a stream receiver that will use clearAll() internally -
> this
> way you will combine performance advantages of the streamer and clearAll()
> method. Here is the sample implementation:
>
> public class ClearAllReceiver implements StreamReceiver<MyKey, MyClass> {
>     @Override public void receive(IgniteCache<MyKey, MyClass> cache,
> Collection<Entry&lt;MyKey, MyClass>> entries) {
>         Set<MyKey> keys = new HashSet<>(entries.size());
>
>         for (Entry<MyKey, MyClass> entry : entries)
>             keys.add(entry.getKey());
>
>         cache.clearAll(keys);
>     }
> }
>
> Let me know if anything is unclear.
>
> -Val
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/DataStreamer-performance-with-removing-data-tp1917p1923.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Mime
View raw message