cayenne-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Reich <markus.re...@markusreich.at>
Subject Re: Fetching lots of objects
Date Tue, 07 Mar 2017 07:30:25 GMT
Hi Maik,

maybe you can use the new iterator and split the iterator for parallel
computation?

public static <T> Stream<T> asStream(Iterator<T> sourceIterator, boolean
parallel) {
   Iterable<T> iterable = () -> sourceIterator;
   return StreamSupport.stream(iterable.spliterator(), parallel);
}

found at
http://stackoverflow.com/questions/24511052/how-to-convert-an-iterator-to-a-stream

br
Meex

Musall, Maik <maik@selbstdenker.ag> schrieb am Mo., 6. März 2017 um
22:25 Uhr:

> Hi all,
>
> I have a number of statistics functions which need to fetch large amounts
> of objects. I need the actual DataObjects because that's where the business
> logic is that I need for the computations.
>
> Let's say I need to fetch 300.000 objects. Let's also assume the database
> sits on a fast SSD array and can serve multiple connections easily. I'm
> assuming in this case the CPU time needed for DataObject instantiation is
> the main performance constraint. Is that correct?
>
> If so, how can I speed this up? Could I partition my fetch, and fetch in
> several threads in parallel into the same ObjectContext? Or is there an
> easier way to make use of multiple CPU cores for this?
>
> Thanks
> Maik
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message