cayenne-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrus Adamchik <>
Subject Re: paged query slow when fetching big lists
Date Sat, 23 Jun 2007 07:38:34 GMT
Ari, Marcin --

going through the code I noticed one inefficiency - the elements  
array access is synchronized in 'fillIn' method. Since 'fillIn' is  
called from constructor, such synchronization is unneeded and only  
slows things down. I just checked a fixed version to trunk. Could you  
try it out?


On Jun 23, 2007, at 12:33 AM, Aristedes Maniatis wrote:
> On 22/06/2007, at 11:10 PM, Michael Gentry wrote:
>> Marcin, this thread might be of interest to you ...
>> browser
>> Look at the "Paging and SQL queries" thread on May 25.
> Yes, this is the same project we are working on. I started some  
> performance profiling and Marcin has been able now to take it much  
> further. What is it about:
>> elements.add(it.nextObjectId(entity));
> which is so slow? The code gets a little complex at that point and  
> we are having difficulty tracing it through to the exact  
> performance problem in the underlying code. Is it the speed of  
> adding the object id to the Collection or the speed of creating an  
> object id itself? 0.5ms doesn't sound slow, but it doesn't scale well.
> Andrus, I got the impression from the previous thread that you  
> suspected something slightly different. That the performance  
> problem might be in the fat query itself, but from our tests this  
> isn't the case. If I've got this right, the way it works is:
> 1. perform regular query to get all columns but return result in  
> iterator
> 2. iterate through first page and build full objects
> 3. iterate through other pages and build just objectids (this is  
> the slow part for us)
> 4. when another page is fetched perform another query and fetch  
> those objects from the DB
> So, getting just primary keys from the DB may not be any faster if  
> the performance problem is simply in the construction of objectIds.  
> If the performance problem is in the numerous resizings of the  
> Collection (each time it runs out of space, then it is increased by  
> 50% or 100% in size), then the solution could be as simple as  
> figuring out the size of the iterator and sizing the collection  
> appropriately from the beginning.
> Any ideas on how to discover the exact cause of the performance hit?
> Ari Maniatis
> -------------------------->
> ish
> Level 1, 30 Wilson Street Newtown 2042 Australia
> phone +61 2 9550 5001   fax +61 2 9550 4001
> GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

View raw message