openjpa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dimitrios Iosifidis (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (OPENJPA-2078) Using external datacache - poor performance when trying to fetch a large resultSet
Date Tue, 22 Nov 2011 14:26:40 GMT
Using external datacache - poor performance when trying to fetch a large resultSet
----------------------------------------------------------------------------------

                 Key: OPENJPA-2078
                 URL: https://issues.apache.org/jira/browse/OPENJPA-2078
             Project: OpenJPA
          Issue Type: Question
          Components: datacache, performance
    Affects Versions: 2.1.0
         Environment: Mysql, memcached, SLES 11
            Reporter: Dimitrios Iosifidis
            Priority: Critical


In our application, we have extended the AbstractDataCache/AbstractQueryCache in order to
introduce our own caching implementation using Memcached.

However, during testing it has been found that performance has dramatically degraded. Analysis
has shown that the main reason for this is the fact that openjpa calls the
 
protected DataCachePCData getInternal(final Object arg0)  

method instead (probably) of the:

Map<Object, DataCachePCData> getAll(List<Object> arg0)


For a result set of 40000 records for example, 40000 get requests are issued to the external
cache, resulting in poor performance. Instead, if the getAll(..) method (overrided from AbstractDataCache)
was to be called, then the cache would have a decent performance (as a consequence of a single
request).

So, my question consists of the following:
1) Is there a relevance between query cache and data cache? The query I am experimenting on
is like:
SELECT user FROM UserPO AS user WHERE user.domain.fname = :domainName
for which I get at the console output:

"WARN  [Runtime] Query "SELECT user FROM UserPO AS user WHERE user.domain.fname = :domainName"
is removed from cache  excluded permanently. Query "SELECT user FROM UserPO AS user WHERE
user.domain.fname = :domainName" is not cached because its result is not obtained by executing
a select statement. This can happen if the query was evaluated in-memory. The result was provided
by org.apache.openjpa.datacache.QueryCacheStoreQuery$CachingResultObjectProvider. "

This means subsequently that this type of query cannot benefit from caching techniques? 

2) Is there a way to get all ids of the PO's from openjpa that consist the resultlist of a
specific query? 

3) In this particular example, enabling the default cache (openJPA's):
 <property name="openjpa.DataCache" value="true"/> 
 <property name="openjpa.QueryCache" value="true"/> 
seems to work fine (in respect of performance) 
The above queries are issued two times, but only one reached the database. Taking into account
that the same warning is displayed (Query "SELECT user FROM UserPO AS user WHERE user.domain.fname
= :domainName" is removed from cache  excluded permanently.....) how this works?

4) Is there a mechanism to trigger using the "bulk" getAll, loadAll() etc methods?

Any hints are highly appreciated! 

Thank you in advance, 
Dimitris.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message