cayenne-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aristedes Maniatis <...@maniatis.org>
Subject Re: Fetching lots of objects
Date Tue, 07 Mar 2017 22:14:20 GMT
On 7/3/17 8:25am, Musall, Maik wrote:
> Hi all,
> 
> I have a number of statistics functions which need to fetch large amounts of objects.
I need the actual DataObjects because that's where the business logic is that I need for the
computations.
> 
> Let's say I need to fetch 300.000 objects. Let's also assume the database sits on a fast
SSD array and can serve multiple connections easily. I'm assuming in this case the CPU time
needed for DataObject instantiation is the main performance constraint. Is that correct?
> 
> If so, how can I speed this up? Could I partition my fetch, and fetch in several threads
in parallel into the same ObjectContext? Or is there an easier way to make use of multiple
CPU cores for this?


I don't think there is anything in Cayenne that will specifically help you here. However if
you can partition your search query, the of course you can fetch the data in multiple threads
in parallel.

You might also want to fetch into DataRows rather than creating object entities. I'm not sure
if that will make your use case faster, but you could try, especially if you don't need all
the columns from the db entity.

Ari



-- 
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

Mime
View raw message