camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gonzalo Vasquez <gvasq...@altiuz.cl>
Subject Re: Camel performance tuning
Date Fri, 09 Nov 2012 16:49:34 GMT
That sounds much better, but how do I write that in spring DSL?

Gonzalo Vásquez Sáez
Gerente Investigación y Desarrollo (R&D)
Altiuz Soluciones Tecnológicas de Negocios Ltda.
Av. Nueva Tajamar 555 Of. 802, Las Condes
(56-2) 335 2461
gvasquez@altiuz.cl
http://www.altiuz.cl
 


El 09-11-2012, a las 13:43, Claus Straube <claus.straube@catify.com> escribió:

> Hi Gonzales. Take a look at this: http://www.catify.com/2012/07/09/parsing-large-files-with-apache-camel/
perhaps it solves your issue.
> 
> Best regards - Claus
>> 
>> I'm running a route that basically adds a character per line to a plain text file,
but it's taking to long, and it seems that it's due to some kind of buffering issue when reading/writing
from disk.
>> 
>> I'm processing a 5MB file (attached as DC_FACCL132_0000 MORA_1075_16-10-2012_19-09-47_15.txt.zip),
with the corresponding XSL template (also attached).
>> 
>> It's taking for ever to process such a file, I understand I'm tokenizing on line
breaks, which could be the source of the problem as there are many lines in the file (48198
exactly), but when running jvisualvm (see attached images/snapshot)I can see the writing op
is invoked 20386 times, which seem not related to the line count. Is there an output buffer
size that I can configure? Or something like that?
>> 
>> This is the route:
>> 
>> <camel:routeid="pager"autoStartup="true">
>> 
>> <camel:from
>> 
>> uri="file:///tmp/in?charset=Windows-1252&amp;move=${file:parent}/../paged/${file:name.noext}.paged.ack&amp;preMove=${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.${file:ext}
<file:///%5C%5Ctmp%5Cin?charset=Windows-1252&amp;move=$%7bfile:parent%7d/../paged/$%7bfile:name.noext%7d.paged.ack&amp;preMove=$%7bfile:name.noext%7d-$%7bdate:now:yyyyMMddHHmmssSSS%7d.$%7bfile:ext%7d>"/>
>> 
>> <camel:splitstreaming="true"parallelProcessing="false">
>> 
>> <camel:tokenizetoken="\n"/>
>> 
>> <camel:touri="bean:pager"/>
>> 
>> <camel:to
>> 
>> uri="file:///tmp/paged?charset=utf8&amp;fileName=${file:name.noext}.paged&amp;fileExist=Append
<file:///%5C%5Ctmp%5Cpaged?charset=utf8&amp;fileName=$%7bfile:name.noext%7d.paged&amp;fileExist=Append>"/>
>> 
>> </camel:split>
>> 
>> </camel:route>
>> 
>> This is the referenced bean:
>> 
>> <beanid="pager"class="cl.altiuz.reports.etl.TextProcessor">
>> 
>> <propertyname="xsltPath"
>> 
>> value="/Users/gonzalovasquez/Documents/workspace/altiuz-reports/reports-etl/xsl/pager.xsl"/>
>> 
>> <propertyname="param"value="C.*PAG.* 1"/>
>> 
>> </bean>
>> 
>> Camel versión is 2,10.1, and happens both on OSX & MS Windows, so I think isn't
a platform dependent problem, but a configuration one.
>> 
>> Any ideas? Any thing else that I should send?
>> 
>> Thanks!
>> 
>> *Gonzalo Vásquez Sáez*
>> 
>> *Gerente Investigación y Desarrollo (R&D)*
>> *Altiuz* Soluciones Tecnológicas de Negocios Ltda.
>> Av. Nueva Tajamar 555 Of. 802, Las Condes
>> (56-2) 335 2461
>> _gvasquez@altiuz.c <mailto:gcoppa@altiuz.com>l_
>> 
>> _http://www.altiuz.cl <http://www.altiuz.cl/>_
>> 
>> This e-mail and any files transmitted with it are for the sole use of the intended
recipient(s) and may contain confidential and privileged information. If you are not the intended
recipient(s), please reply to the sender and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this
email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited
and may be unlawful. 


Mime
View raw message