Mailing-List: contact user-help@cayenne.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cayenne.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Message-ID: <50C914E9.9090305@maniatis.org>
Date: Thu, 13 Dec 2012 10:36:09 +1100
From: Aristedes Maniatis <ari@maniatis.org>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8;
 rv:18.0) Gecko/20121128 Thunderbird/18.0
MIME-Version: 1.0
To: user@cayenne.apache.org
Subject: Re: Fetching and processing a large amount of objects
References: <4D7F03C0-66B2-4062-B421-E96EC6B0BC9C@mackoy.de>
In-Reply-To: <4D7F03C0-66B2-4062-B421-E96EC6B0BC9C@mackoy.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 13/12/12 12:18am, Simon Schneider wrote:
> Hi list,
>
> my task is to fetch and process a large amount of objects. Reading the documentation my best choice
> seemed to do a dataContext.performIteratedQuery(query). The problem is that I get a Java Heap Space
> error, on executing this statement. Now it seems to me the problem is that performIteratedQuery loads
> all data into a ResultSet.
> My question is how to handle such cases, is there any best practice? Would it be ok to use a paginated
> query. Then unregistering processed objects from the DataContext they belong to and also removing
> them from the array that resulted from the paginated query.

By the time you step through to the end of the results, even with a paginated query you will have read every object into RAM. However paginated queries are a really convenient way to fetch even millions of rows where you don't necessarily want to access every object (eg. when you want to display a scrollable list of results and you only need to resolve the objects you can see in the current viewport).

As Michael says, an option is to read in the data in batches into different Contexts.

Ari
  

-- 
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A