cayenne-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aristedes Maniatis <...@maniatis.org>
Subject Re: Object Caching
Date Fri, 13 Nov 2009 11:44:46 GMT
On 13/11/09 10:04 PM, Hans Pikkemaat wrote:
> I ran some tests using 3.0b with SQLTemplate in combination with
> prefetching and found
> a possible new problem.
>
> It seems that when running the query in eg 1 minute, it takes about 2
> minutes before cayenne
> has constructed the prefetched objects.
>
> My query produces 2.5 million records. The query will take about 30
> minutes. Construction
> of the objects will then take an extra hour.


Just to be clear about what you are doing:

* Cayenne 3.0 beta 1
* SQL template query
* prefeching across to-many join
* paging on

You expect the first query (which gets hollow objects) to NOT include the prefetch JOIN, but
when fetching a page of results, it should use the prefetch. Cayenne is constructing the first
query which includes the JOIN and that makes it take 30 minutes in your database to return
2.5 million records.

Is that correct?


My opinions:

1. Does prefetch really help here anyway? You are only getting (say) 100 records at a time,
so the extra queries to follow the relations may not be that significant.

2. Do you really want to fetch 2.5 million rows? If so, I assume this is not a user interface
:-)  Perhaps Cayenne (or any ORM for that matter) is not the best way to batch process that
many rows.



Ari

-- 

-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

Mime
View raw message