db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edson Carlos Ericksson Richter <edson.rich...@mgrinformatica.com.br>
Subject Re: Iterate over big table...
Date Fri, 19 May 2006 13:34:16 GMT
Have you read the pattern "ValueListHandler". It's a kind of iterator 
pattern for large resultsets. It's commonly used by apps to lazy 
materialization... I've been using it with success, with tables around 
700K records (without the pattern, even 700K records throws OOME on 
standard 64Mb JVM).

Of course, you could extends JVM memory (parameter -Xmx512M, to use 
512Mb), but pattern solution is more charming.

Richter


yves pielusenet escreveu:
> Hello,
> I have such a table :
> CREATE TABLE data (
>    numvign INTEGER PRIMARY KEY,
>    data BLOB NOT NULL
> )
>
> This table should have millions of records. I have to iterate over all
> records.
> What is the best way (I don't want to have a outOfMemoryException) :
>
> for(int i=0; i<nbRows; i++){
>     theData = select data from data where numvign=i;
>     doSomething(theData);
> }
>
> or :
>
> alldata = select data from data order by numvign;
> while(alldata.hasNext()){
>    doSomething(allData.next());
> }
>
> When I do a big select, are all selected rows into memory or derby
> access to database when it necessary ?
>
> thanks :)
>
>   


Mime
View raw message