camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Spliter in Camel
Date Thu, 27 Oct 2011 05:24:19 GMT
On Wed, Oct 26, 2011 at 9:52 PM, ebinsingh
<ebenezer.singh@verizonwireless.com> wrote:
> Hi,
>
> I have a file of 300,000 records and i use the split mechanism of Camel to
> split them and sends each record to a processer.
> Does Camel store these records on a heap or somewhere before it sends them
> to the processer. How does Camel splitter internally work.
>
> I want to make sure that the Splitter does not eat up the memory.
>

The splitter supports a .streaming() option that causes it to split it
in chunks.
That requires thought the source message can support this.
http://camel.apache.org/splitter

For example to split a CSV file on a line by line basis in streaming
mode, will not load
the entire file into memory, so you can split very big files this way.

XPath as Christian said could potential cause the message to be loaded
into memory.
Since we recent have optimized XQuery and XSLT, we are looking into
optimize XPath as well if that is possible.


> Please provide some insight into this.
>
> Thanks & regards,
> Ebe
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Spliter-in-Camel-tp4940967p4940967.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/

Mime
View raw message