camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dwarak_cit <>
Subject Large file processing using ExecutorService and IBM MQ
Date Tue, 30 Sep 2014 03:10:55 GMT
I have a requirement where I need to read quite a lot of large files. For
each file, store each record by record (separated by new line delimiter)
into the database. Now I have used ExecutorService with a fixed thread pool
of 100, in Camel split component to split the file (say has 20,000 records)
and send 200 messages per thread to IBM MQ Websphere. 

Now I created another processor which reads these messages from the IBM's
Message Queue. Here I have configured as
/*<bean id="websphere" class="org.apache.camel.component.jms.JmsComponent">
<property name="concurrentConsumers" value="200" />
<property name="maxConcurrentConsumers" value="200" />
<property name="taskExecutor" ref="taskExecutor" />
* /

1) Does this mean there are 200 threads to consume 200 message?
2) Need some more information on ExecutorService and Concurrent Consumers.

Also, this configuration would invoke 200 parallel message consumption. But
how much ever i increase my thread size and concurrent consumers, my
processing speed is slower than a stand alone program which processes that
file spawning multiple threads.


View this message in context:
Sent from the Camel Development mailing list archive at

View raw message