ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srikanta Patanjali <p.srika...@gmail.com>
Subject Increase the indexing speed while loading the cache from an RDBMS
Date Wed, 26 Aug 2020 15:47:25 GMT
Currently I'm using Apache Ignite v2.8.1 to preload a cache from the RDBMS.
There are two tables with each 27M rows. The index is defined on a single
column of type String in 1st table and Integer in the 2nd table. Together
the total size of the two tables is around 120GB.

The preloading process (triggered using loadCacheAsync() from within a Java
app) takes about 45hrs. The cache is persistence enabled and a common EBS
volume (SSD) is being used for both the WAL and other locations.

I'm unable to figure out the bottleneck for increasing the speed.

Apart from defining a separate path for WAL and the persistence, is there
any other way to load the cache faster (with indexing enabled) ?


Thanks,
Srikanta

Mime
View raw message