ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Kuznetsov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-4164) Add support for parallel loading of caches using custom SQL load queries in org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore#loadCache
Date Thu, 03 Nov 2016 08:23:58 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15632037#comment-15632037
] 

Alexey Kuznetsov commented on IGNITE-4164:
------------------------------------------

I think it will be very hard to implement.

But {{CacheAbstractJdbcStore}} already can process custom SQL queries in parallel.

You could do it from user code like this:
{code}
  cache.loadCache(null,
     "someKeyClass", "select * from SomeTable where id < 10000",
     "someKeyClass", "select * from SomeTable where id >= 10000 and id < 20000",
.....
     "someKeyClass", "select * from SomeTable where id >= XXXXX and id < YYYY")
{code}

All listed queries will be executed in parallel.

You may build such list of queries as array and pass it to  {{cache.loadCache()}} because
 {{cache.loadCache()}} required second argument as {{Object ...args}}

If this work for you, please close this issue as {{Won't Fix}}.


> Add support for parallel loading of caches using custom SQL load queries in org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore#loadCache
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-4164
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4164
>             Project: Ignite
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 1.7
>            Reporter: Anghel Botos
>
> Please add support for parallel loading of caches using custom SQL load queries in org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore#loadCache.
For the moment this is not possible, as the current implementation performs the load for each
entity type in the cache using a single {{LoadCacheCustomQueryWorker}}, as opposed to the
the approach that is used when no custom SQL queries are provided for the load, where for
each entity type, the load is distributed across several threads based on some ranges.
> While it may not be possible to support parralel load with any custom SQL query (as this
would mean that Ignite would have to somehow understand the meaning of that custom query),
it would still be a significant improvement if it would be possible to have the parallel load
when providing a custom {{WHERE}} clause for each entity type (instead of a full custom query).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message