camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Straube <claus.stra...@catify.com>
Subject Re: Camel performance tuning
Date Fri, 09 Nov 2012 16:55:51 GMT
Sound simple but take a look at: http://camel.apache.org/aggregator2.html ;)

On 09.11.2012 17:49, Gonzalo Vasquez wrote:
> That sounds much better, but how do I write that in spring DSL?
>
> Gonzalo Vásquez Sáez
> Gerente Investigación y Desarrollo (R&D)
> Altiuz Soluciones Tecnológicas de Negocios Ltda.
> Av. Nueva Tajamar 555 Of. 802, Las Condes
> (56-2) 335 2461
> gvasquez@altiuz.c <mailto:gvasquez@altiuz.c>l
> http://www.altiuz.cl
>
>
>
> El 09-11-2012, a las 13:43, Claus Straube <claus.straube@catify.com 
> <mailto:claus.straube@catify.com>> escribió:
>
>> Hi Gonzales. Take a look at this: 
>> http://www.catify.com/2012/07/09/parsing-large-files-with-apache-camel/ 
>> perhaps it solves your issue.
>>
>> Best regards - Claus
>>>
>>> I'm running a route that basically adds a character per line to a 
>>> plain text file, but it's taking to long, and it seems that it's due 
>>> to some kind of buffering issue when reading/writing from disk.
>>>
>>> I'm processing a 5MB file (attached as DC_FACCL132_0000 
>>> MORA_1075_16-10-2012_19-09-47_15.txt.zip), with the corresponding 
>>> XSL template (also attached).
>>>
>>> It's taking for ever to process such a file, I understand I'm 
>>> tokenizing on line breaks, which could be the source of the problem 
>>> as there are many lines in the file (48198 exactly), but when 
>>> running jvisualvm (see attached images/snapshot)I can see the 
>>> writing op is invoked 20386 times, which seem not related to the 
>>> line count. Is there an output buffer size that I can configure? Or 
>>> something like that?
>>>
>>> This is the route:
>>>
>>> <camel:routeid="pager"autoStartup="true">
>>>
>>> <camel:from
>>>
>>> uri="file:///tmp/in?charset=Windows-1252&amp;move=${file:parent}/../paged/${file:name.noext}.paged.ack&amp;preMove=${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.${file:ext}

>>> <file:///tmp/in?charset=Windows-1252&amp;move=$%7Bfile:parent%7D/../paged/$%7Bfile:name.noext%7D.paged.ack&amp;preMove=$%7Bfile:name.noext%7D-$%7Bdate:now:yyyyMMddHHmmssSSS%7D.$%7Bfile:ext%7D>

>>> <file:///%5C%5Ctmp%5Cin?charset=Windows-1252&amp;move=$%7bfile:parent%7d/../paged/$%7bfile:name.noext%7d.paged.ack&amp;preMove=$%7bfile:name.noext%7d-$%7bdate:now:yyyyMMddHHmmssSSS%7d.$%7bfile:ext%7d>"/>
>>>
>>> <camel:splitstreaming="true"parallelProcessing="false">
>>>
>>> <camel:tokenizetoken="\n"/>
>>>
>>> <camel:touri="bean:pager"/>
>>>
>>> <camel:to
>>>
>>> uri="file:///tmp/paged?charset=utf8&amp;fileName=${file:name.noext}.paged&amp;fileExist=Append

>>> <file:///tmp/paged?charset=utf8&amp;fileName=$%7Bfile:name.noext%7D.paged&amp;fileExist=Append>

>>> <file:///%5C%5Ctmp%5Cpaged?charset=utf8&amp;fileName=$%7bfile:name.noext%7d.paged&amp;fileExist=Append>"/>
>>>
>>> </camel:split>
>>>
>>> </camel:route>
>>>
>>> This is the referenced bean:
>>>
>>> <beanid="pager"class="cl.altiuz.reports.etl.TextProcessor">
>>>
>>> <propertyname="xsltPath"
>>>
>>> value="/Users/gonzalovasquez/Documents/workspace/altiuz-reports/reports-etl/xsl/pager.xsl"/>
>>>
>>> <propertyname="param"value="C.*PAG.* 1"/>
>>>
>>> </bean>
>>>
>>> Camel versión is 2,10.1, and happens both on OSX & MS Windows, so I 
>>> think isn't a platform dependent problem, but a configuration one.
>>>
>>> Any ideas? Any thing else that I should send?
>>>
>>> Thanks!
>>>
>>> *Gonzalo Vásquez Sáez*
>>>
>>> *Gerente Investigación y Desarrollo (R&D)*
>>> *Altiuz* Soluciones Tecnológicas de Negocios Ltda.
>>> Av. Nueva Tajamar 555 Of. 802, Las Condes
>>> (56-2) 335 2461
>>> _gvasquez@altiuz.c <mailto:gvasquez@altiuz.c> 
>>> <mailto:gcoppa@altiuz.com>l_
>>>
>>> _http://www.altiuz.cl <http://www.altiuz.cl/>_
>>>
>>> This e-mail and any files transmitted with it are for the sole use 
>>> of the intended recipient(s) and may contain confidential and 
>>> privileged information. If you are not the intended recipient(s), 
>>> please reply to the sender and destroy all copies of the original 
>>> message. Any unauthorized review, use, disclosure, dissemination, 
>>> forwarding, printing or copying of this email, and/or any action 
>>> taken in reliance on the contents of this e-mail is strictly 
>>> prohibited and may be unlawful.
>

Mime
View raw message