camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem jiang <willem.ji...@gmail.com>
Subject Re: Large file processing with Apache Camel
Date Fri, 22 Feb 2013 07:18:27 GMT
Hi,

You can take a look at the VM component[1] or NMR component[2] to leverage the multiple thread
to deal the XML processing.

[1]http://camel.apache.org/vm.html
[2]http://camel.apache.org/nmr.html

--  
Willem Jiang

Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) (English)
          http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang  
Weibo: 姜宁willem





On Friday, February 22, 2013 at 1:21 PM, cristisor wrote:

> Many thanks for your reply.
>  
> When I read from the file I read simple lines, not XML. It takes more than
> one hour to read, process and insert into the db 20.000 lines so I took out
> the service unit that does the db operations and I was left with 26 minutes
> for reading from the file line by line, converting to a stage 1 XML and then
> to a stage 2 XML. The machine is an i7 with 8 GB of RAM, so I don't think
> that it's normal to take so long. This is why I suspected that the I/O
> operations, when reading the file, are slowing me down very much.
>  
> Do you think that it will take me 100 times more to read 100 lines instead
> of just one? The performance hit might not be at this point?
>  
> Is sending the exchange to the queue and reading it inside another service
> unit slowing me down?
>  
> I'm not using Stax Stream, or at least I don't think that I'm using, to deal
> the XML but I will take a look on this problem also.
>  
>  
>  
> --
> View this message in context: http://camel.465427.n5.nabble.com/Large-file-processing-with-Apache-Camel-tp5727977p5727988.html
> Sent from the Camel - Users mailing list archive at Nabble.com (http://Nabble.com).




Mime
View raw message