camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem jiang <>
Subject Re: Large file processing with Apache Camel
Date Fri, 22 Feb 2013 07:18:27 GMT

You can take a look at the VM component[1] or NMR component[2] to leverage the multiple thread
to deal the XML processing.


Willem Jiang

Red Hat, Inc.
FuseSource is now part of Red Hat
Web: |
Blog: ( (English)
 ( (Chinese)
Twitter: willemjiang  
Weibo: 姜宁willem

On Friday, February 22, 2013 at 1:21 PM, cristisor wrote:

> Many thanks for your reply.
> When I read from the file I read simple lines, not XML. It takes more than
> one hour to read, process and insert into the db 20.000 lines so I took out
> the service unit that does the db operations and I was left with 26 minutes
> for reading from the file line by line, converting to a stage 1 XML and then
> to a stage 2 XML. The machine is an i7 with 8 GB of RAM, so I don't think
> that it's normal to take so long. This is why I suspected that the I/O
> operations, when reading the file, are slowing me down very much.
> Do you think that it will take me 100 times more to read 100 lines instead
> of just one? The performance hit might not be at this point?
> Is sending the exchange to the queue and reading it inside another service
> unit slowing me down?
> I'm not using Stax Stream, or at least I don't think that I'm using, to deal
> the XML but I will take a look on this problem also.
> --
> View this message in context:
> Sent from the Camel - Users mailing list archive at (

View raw message