cayenne-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <do...@xsinet.co.za>
Subject Re: Cayenne and threading
Date Fri, 19 Jun 2015 05:55:29 GMT
I've just done something like this, along these lines:

    private class ProcessImages extends Task<Void>
    {
        private int imageCount = 0;
        private int  listSize;

        @Override
        protected Void call() throws Exception
        {
            String  sql = ....
            SQLTemplate  imgQry = new SQLTemplate( ImageMeta.class, sql );
            imgQry.setCacheStrategy( QueryCacheStrategy.NO_CACHE );
            imgQry.setFetchingDataRows( true );

            List<DataRow>  imgList = getNewContext().performQuery( imgQry );
            listSize = imgList.size();

            imgList.parallelStream().forEach( this::processImage );

            return null;
        }

        private void processImage( DataRow imgID )
        {
            updateProgress( ++imageCount, listSize );

            ImageBlob  imgBlob = Cayenne.objectForPK( getNewContext(), 
ImageBlob.class, imgID.get( "MED_IM_ID" ) );

            //  do stuff to imgBlob

            imgBlob.getObjectContext().commitChanges();
        }

    }


regards
Jurgen


-----Original Message----- 
From: Hugi Thordarson
Sent: Friday, June 19, 2015 12:32 AM
To: user@cayenne.apache.org
Subject: Re: Cayenne and threading

“Contexts are cheap”. Word’s I’m living by from now on, thanks!

- hugi



> On 18. jún. 2015, at 22:29, John Huss <johnthuss@gmail.com> wrote:
>
> Contexts are cheap. You can create a new one on each iteration (or bind 
> one
> to each thread ahead of time and reuse them) and localize the object and
> save. Or do the write as plain sql.
> On Thu, Jun 18, 2015 at 5:17 PM Hugi Thordarson <hugi@karlmenn.is> wrote:
>
>> Thanks for the reply John, I suspected it couldn’t really be *this* good
>> :).
>>
>> But if there are any takers, I’d love to hear how experienced folks would
>> approach multithreaded DB writes for tasks such as this using Cayenne.
>>
>> To make things more clear; what’s happening in the lambda in my actual
>> project is that I’m performing OCR on an image linked to each Receipt
>> object and storing the resulting text. So, basically; a long running 
>> task,
>> performing an operation on fetched objects and updating them.
>>
>> When using EOF I’ve been bypassing DB writes entirely during the
>> multithreaded OCR stage by writing OCR results to disk. Once OCR is
>> complete for all receipts, I read the text files back form disk and 
>> update
>> the DB.
>>
>> Optimally, I’d like to be able to update the objects (and save to the DB)
>> as I’m iterating through the receipt objects. Any ideas?
>>
>> Cheers,
>> - hugi
>>
>>
>>
>>> On 18. jún. 2015, at 21:48, John Huss <johnthuss@gmail.com> wrote:
>>>
>>> No, I don't think that's a good idea.  The same context shouldn't be
>> shared
>>> among threads.  If you're doing pure SQL operations (like SQLTemplate)
>> and
>>> not touching DataObjects you may be ok, but otherwise not.
>>>
>>> But yes, Executors are great, especially with lambdas.
>>>
>>> John
>>>
>>> On Thu, Jun 18, 2015 at 4:31 PM Hugi Thordarson <hugi@karlmenn.is>
>> wrote:
>>>
>>>> Hi all.
>>>>
>>>> Coming from EOF I’ve been mentally trained to believe anything threaded
>>>> will bite you in the ass in a bad way. Using Cayenne, I’m now writing
>>>> threaded code and loving it, but I just wanted to double check with the
>>>> community; Is it really OK for me to do stuff like I’m doing in the
>>>> following code? It’s a somewhat extreme example since I'd never 
>>>> actually
>>>> commit for each iteration, just looking for some good practices, advice
>> or
>>>> an “ARE YOU CRAZY?” comment :).
>>>>
>>>> public void main( String[] argv ) {
>>>>       ObjectContext oc = // get my ObjectContext
>>>>
>>>>       ExecutorService executorService = Executors.newFixedThreadPool(
>>>> Runtime.getRuntime().availableProcessors() );
>>>>
>>>>       List<Receipt> receipts = oc.select( new SelectQuery<>(
>>>> Receipt.class ) );
>>>>
>>>>       for( Receipt receipt : receipts ) {
>>>>               executorService.execute( () -> {
>>>>                       receipt.setCreationDate( new Date() );
>>>>                       oc.commitChanges();
>>>>               } );
>>>>       }
>>>>
>>>>       executorService.shutdown();
>>>> }
>>>>
>>>> Cheers,
>>>> - hugi
>>>>
>>>> // Hugi Thordarson
>>>> // http://www.loftfar.is/ <http://www.loftfar.is/>
>>>> // s. 895-6688
>>>>
>>>>
>>
>>


Mime
View raw message