camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem Jiang <>
Subject Re: Grouping lines while streaming
Date Mon, 20 Feb 2012 07:55:07 GMT
It could more easy for user to group the lines if camel-stream component 
can expose the strategy processLine to the client.
I just fill a JIRA[1] for it

You may need to extends the StreamConsumer yourself to implement such of 
feature at this time.


On Mon Feb 20 08:59:09 2012, Anurag Sharma wrote:
> Hi All,
> I have a file with million plus records. Each record is of CSV format. Now
> instead of reading the whole file in memory I would like to stream it record
> by record , or prefereably bunch N nbr of records in a single message
> exchange.
> Following is my route which works OK for streaming it line by line.
> from("file:src/data?fileName=webservices_20090723_001_trunc.log&noop=true").split(body().tokenize("\n")).streaming()
>          	  .to("seda:input?concurrentConsumers=1")
>      	  .log("Processing ${id}");  	
>      	from("seda:input?concurrentConsumers=1")
>      	  .convertBodyTo(String.class)
>      	  .unmarshal(csv)
>    	  .to("bean:LogService?method=doHandleCsvData");
> Now if i want to bunch a group of lines together, it appears that there is
> no group option while streaming from file component. So I decided to try the
> streaming component as follows:
> from("stream:file?fileName=src/data/webservices_20090723_001_trunc.log&groupLines=2")
> Now this does group two lines together however it removes the new line
> seperator. Consequently two records are concatenated in a single list entry
> when the message arrives at doHandleCsvData.
> I suppose I can write my own producer within the File component that takes
> the file handle and streams data out. However I am keen on exploring the
> capabilities of the existring components.
> Would appreciate any help.
> Thanks&  Regards,
> Anurag
> --
> View this message in context:
> Sent from the Camel - Users mailing list archive at

Blog: (English)
Twitter: willemjiang
Weibo: willemjiang

View raw message