ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anand Vijai <anand.vi...@gmail.com>
Subject Retrieving keys very slow from a partitioned, no backup, no replicated cache
Date Tue, 24 Oct 2017 16:40:52 GMT
I have a cache with approx 1MM rows and is partitioned across 3 server nodes.
I am trying to retrieve all the keys from the cache and then trying to
perform an update operation using cache.invoke.
The cache.invoke itself is very fast but the retrieval of the keys takes a
lot of time - approximately 10-100 keys per second is the throughput I am
The cache has 15 fields and 4 of them are defined as key fields and are
indexed but since I am using Scanquery i am guessing that the indexes arent

What is the best/correct way to retrieve all keys and perform an update to
the cache within the server nodes without returning any data back to the
client node. 

Originally i had iterated through the iterator one key at a time and thought
the update operation was taking time. But i changed the code to create a
keyset first and then did a cache.invoke in separate blocks of code as

        // Execute the query.
        Iterator<Cache.Entry&lt;FactKey, Fact>> iterator =
        Set<FactKey> KEYS_SET = new HashSet<FactKey>();
        FactKey key = new FactKey();
        Fact val = null;
        ExecutionTimer t = new ExecutionTimer(); // this is just to track
time taken in seconds

        for (int i = 0; i < cache.size(CachePeekMode.ALL); i++) {
            //while (iterator.hasNext()) {
            key = iterator.next().getKey();


        if(i%1000 == 0)
            System.out.println(">>> Update Count: " + i);
            System.out.println("Time taken for: " + i + " " + t.duration());
            cache.invokeAll(KEYS_SET, (entry, args) -> {
                val = entry.getValue();
                val.setAmount(val.getAmt1() + val.getAmt2());
                return null;
            System.out.println(">>> Updating ADB: " + i);
            System.out.println("Timer for cache invoke: " + t.duration());

Sent from: http://apache-ignite-users.70518.x6.nabble.com/

View raw message