camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem Jiang <willem.ji...@gmail.com>
Subject Re: Grouping lines while streaming
Date Mon, 20 Feb 2012 07:55:07 GMT
It could more easy for user to group the lines if camel-stream component 
can expose the strategy processLine to the client.
I just fill a JIRA[1] for it

You may need to extends the StreamConsumer yourself to implement such of 
feature at this time.

[1]https://issues.apache.org/jira/browse/CAMEL-5017

On Mon Feb 20 08:59:09 2012, Anurag Sharma wrote:
> Hi All,
>
> I have a file with million plus records. Each record is of CSV format. Now
> instead of reading the whole file in memory I would like to stream it record
> by record , or prefereably bunch N nbr of records in a single message
> exchange.
>
> Following is my route which works OK for streaming it line by line.
>
>
> from("file:src/data?fileName=webservices_20090723_001_trunc.log&noop=true").split(body().tokenize("\n")).streaming()
>          	  .to("seda:input?concurrentConsumers=1")
>      	  .log("Processing ${id}");  	
>      	from("seda:input?concurrentConsumers=1")
>      	  .convertBodyTo(String.class)
>      	  .unmarshal(csv)
>    	  .to("bean:LogService?method=doHandleCsvData");
>
> Now if i want to bunch a group of lines together, it appears that there is
> no group option while streaming from file component. So I decided to try the
> streaming component as follows:
>
>
> from("stream:file?fileName=src/data/webservices_20090723_001_trunc.log&groupLines=2")
>
> Now this does group two lines together however it removes the new line
> seperator. Consequently two records are concatenated in a single list entry
> when the message arrives at doHandleCsvData.
>
> I suppose I can write my own producer within the File component that takes
> the file handle and streams data out. However I am keen on exploring the
> capabilities of the existring components.
>
> Would appreciate any help.
>
> Thanks&  Regards,
>
> Anurag
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Grouping-lines-while-streaming-tp5497878p5497878.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>


-- 
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog:    http://willemjiang.blogspot.com (English)
          http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Weibo: willemjiang

Mime
View raw message