db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <de...@segel.com>
Subject RE: iterating over millions of rows
Date Tue, 20 Jan 2009 13:24:41 GMT

Did you try using an index?

> -----Original Message-----
> From: Brian Peterson [mailto:publicayers@verizon.net]
> Sent: Monday, January 19, 2009 10:20 PM
> To: 'Derby Discussion'
> Subject: iterating over millions of rows
> 
> I have a big table, about 1 million rows, that I'm doing a simple "select
> *"
> over. The table is depressingly simple, basically a big VARCHAR for bit
> data
> that stores some serialized bytes. When I profile using VisualVM it seems
> that it is spending most of its time in
> 
> org.apache.derby.impl.store.raw.data.RAFContainer4.readFully
> 
> If I remember correctly, this gets invoked something like 3000 times. Is
> there anything I can do to speed up iterating over this table? It is
> taking
> about 30s to iterate over the 1 million records, but I could have up to 25
> million.
> 
> It is an embedded db using 10.4, JDK 1.6.0_07, running on a Windows XP SP2
> machine. I have page size set to the max, 32K, and the page case size set
> to
> 6000 pages.
> 
> Is there anything I can do, or have I just run up against how fast Windows
> can read off of the disk?
> 
> Brian
> 
> 




Mime
View raw message