cayenne-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aristedes Maniatis <...@maniatis.org>
Subject Re: Fetching and processing a large amount of objects
Date Wed, 12 Dec 2012 23:36:09 GMT
On 13/12/12 12:18am, Simon Schneider wrote:
> Hi list,
>
> my task is to fetch and process a large amount of objects. Reading the documentation
my best choice
> seemed to do a dataContext.performIteratedQuery(query). The problem is that I get a Java
Heap Space
> error, on executing this statement. Now it seems to me the problem is that performIteratedQuery
loads
> all data into a ResultSet.
> My question is how to handle such cases, is there any best practice? Would it be ok to
use a paginated
> query. Then unregistering processed objects from the DataContext they belong to and also
removing
> them from the array that resulted from the paginated query.

By the time you step through to the end of the results, even with a paginated query you will
have read every object into RAM. However paginated queries are a really convenient way to
fetch even millions of rows where you don't necessarily want to access every object (eg. when
you want to display a scrollable list of results and you only need to resolve the objects
you can see in the current viewport).

As Michael says, an option is to read in the data in batches into different Contexts.

Ari
  

-- 
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

Mime
View raw message