Mailing-List: contact torque-user-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Torque Users List" <torque-user@db.apache.org>
Received-SPF: neutral (herse.apache.org: local policy)
Message-ID: <45F28C30.4030102@apache.org>
Date: Sat, 10 Mar 2007 11:45:04 +0100
From: Thomas Vandahl <tv@apache.org>
User-Agent: Thunderbird 1.5.0.10 (Macintosh/20070221)
MIME-Version: 1.0
To: Apache Torque Users List <torque-user@db.apache.org>
Subject: Re: CriteriaEnumeration
References: 
 <F1EF69E69C1EED4FA3BB295B07664EEE523A4B@srv-master.cadooz-ag.intern>
In-Reply-To: 
 <F1EF69E69C1EED4FA3BB295B07664EEE523A4B@srv-master.cadooz-ag.intern>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Thoralf Rickert wrote:
> Hi!
> 
> I'm not sure, if this was discussed before, but I want to send you this
> "extension". 
> 
> Currently I'm working on somekind of "backup-procedure" that reads some
> data from a database and creates a complex XML file (for example: stores
> with addresses and products). To do this, I have to get all "stores"
> from the database. If there are a lot of stores and I make just a
> doSelect(new Criteria()) I have (of course) trouble with the memory
> (number of stores > 100000).
> 
> I could implement a special algorithm, but I decided to implement a
> mechanism that takes a Criteria and set offset/limit internally in a
> loop. In this way I can get 100 stores with one SELECT and do my work.
> If the "pool" is empty, I can read the next 100 stores.
> 

This is more or less what LargeSelect does. I used to use it for such
things - reading data one page at a time. It provides the additional
advantage of a "read-ahead-cache", means it continues reading your data
in the background while you can process the first page of records.

If you are interested, I created a small LargeSelectHelper class which
allows you to continuously read one page at a time and let LargeSelect
do the "chunking".

Bye, Thomas.


---------------------------------------------------------------------
To unsubscribe, e-mail: torque-user-unsubscribe@db.apache.org
For additional commands, e-mail: torque-user-help@db.apache.org