camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Large file processing using ExecutorService and IBM MQ
Date Tue, 30 Sep 2014 06:02:23 GMT
Hi

Please use the user mailing list / user forum for these kind of questions.
http://camel.apache.org/discussion-forums.html

On Tue, Sep 30, 2014 at 5:10 AM, dwarak_cit <dwarak.nathan@gmail.com> wrote:
> Hi,
> I have a requirement where I need to read quite a lot of large files. For
> each file, store each record by record (separated by new line delimiter)
> into the database. Now I have used ExecutorService with a fixed thread pool
> of 100, in Camel split component to split the file (say has 20,000 records)
> and send 200 messages per thread to IBM MQ Websphere.
>
> Now I created another processor which reads these messages from the IBM's
> Message Queue. Here I have configured as
> /*<bean id="websphere" class="org.apache.camel.component.jms.JmsComponent">
> <property name="concurrentConsumers" value="200" />
> <property name="maxConcurrentConsumers" value="200" />
> <property name="taskExecutor" ref="taskExecutor" />
> * /
>
> 1) Does this mean there are 200 threads to consume 200 message?
> 2) Need some more information on ExecutorService and Concurrent Consumers.
>
> Also, this configuration would invoke 200 parallel message consumption. But
> how much ever i increase my thread size and concurrent consumers, my
> processing speed is slower than a stand alone program which processes that
> file spawning multiple threads.
>
> Thanks!!
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Large-file-processing-using-ExecutorService-and-IBM-MQ-tp5757222.html
> Sent from the Camel Development mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/

Mime
View raw message