ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vkulichenko <valentin.kuliche...@gmail.com>
Subject Re: DataStreamer performance with removing data
Date Tue, 10 Nov 2015 23:29:12 GMT
Hi Kevin,

The difference is that data streamer uses remove() method to update the
cache, while your second test uses clearAll(). remove() and removeAll() are
transactional operations and it also updates persistence store if you have
it. clearAll(), on the other side, only removes the data from the in-memory
cache. It will also skip entries that are currently enlisted in transactions
or locked. To make this comparison fair, you should use removeAll() instead
of clearAll(). Can you try it?

If you're OK with semantics of clearAll() method, you can just use it. You
also can create a stream receiver that will use clearAll() internally - this
way you will combine performance advantages of the streamer and clearAll()
method. Here is the sample implementation:

public class ClearAllReceiver implements StreamReceiver<MyKey, MyClass> {
    @Override public void receive(IgniteCache<MyKey, MyClass> cache,
Collection<Entry&lt;MyKey, MyClass>> entries) {
        Set<MyKey> keys = new HashSet<>(entries.size());

        for (Entry<MyKey, MyClass> entry : entries)
            keys.add(entry.getKey());

        cache.clearAll(keys);
    }
}

Let me know if anything is unclear.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/DataStreamer-performance-with-removing-data-tp1917p1923.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Mime
View raw message