ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Igor Rudyak (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-3962) Cassandra parallel data loader
Date Fri, 23 Sep 2016 23:36:20 GMT
Igor Rudyak created IGNITE-3962:

             Summary: Cassandra parallel data loader
                 Key: IGNITE-3962
                 URL: https://issues.apache.org/jira/browse/IGNITE-3962
             Project: Ignite
          Issue Type: Improvement
          Components: ignite-cassandra
            Reporter: Igor Rudyak
            Assignee: Igor Rudyak

There should be an option to warm up Ignite caches backed by Cassandra using *loadCache* method
and distributing download process among all the nodes of Ignite cluster - downloading portion
of data from Cassandra in parallel. 

Each Ignite node responsible for a set of Ignite cache partitions. By providing some custom
mapping structure to *loadCache* method it's rather easy to map Ignite partitions to "subpartitions"
of Cassandra data slice which we are interested in. Such a way it will be possible to download
all the data in parallel by utilizing all Ignite cluster nodes and running multiple download
thread on each node.

This message was sent by Atlassian JIRA

View raw message