Have you read the pattern "ValueListHandler". It's a kind of iterator
pattern for large resultsets. It's commonly used by apps to lazy
materialization... I've been using it with success, with tables around
700K records (without the pattern, even 700K records throws OOME on
standard 64Mb JVM).
Of course, you could extends JVM memory (parameter -Xmx512M, to use
512Mb), but pattern solution is more charming.
Richter
yves pielusenet escreveu:
> Hello,
> I have such a table :
> CREATE TABLE data (
> numvign INTEGER PRIMARY KEY,
> data BLOB NOT NULL
> )
>
> This table should have millions of records. I have to iterate over all
> records.
> What is the best way (I don't want to have a outOfMemoryException) :
>
> for(int i=0; i<nbRows; i++){
> theData = select data from data where numvign=i;
> doSomething(theData);
> }
>
> or :
>
> alldata = select data from data order by numvign;
> while(alldata.hasNext()){
> doSomething(allData.next());
> }
>
> When I do a big select, are all selected rows into memory or derby
> access to database when it necessary ?
>
> thanks :)
>
>
|