camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anurag Sharma <anurag.das.sha...@gmail.com>
Subject Grouping lines while streaming
Date Mon, 20 Feb 2012 00:59:09 GMT
Hi All,

I have a file with million plus records. Each record is of CSV format. Now
instead of reading the whole file in memory I would like to stream it record
by record , or prefereably bunch N nbr of records in a single message
exchange.

Following is my route which works OK for streaming it line by line.


from("file:src/data?fileName=webservices_20090723_001_trunc.log&noop=true").split(body().tokenize("\n")).streaming()
        	  .to("seda:input?concurrentConsumers=1")
    	  .log("Processing ${id}");  	
    	from("seda:input?concurrentConsumers=1")
    	  .convertBodyTo(String.class)
    	  .unmarshal(csv)
  	  .to("bean:LogService?method=doHandleCsvData");

Now if i want to bunch a group of lines together, it appears that there is
no group option while streaming from file component. So I decided to try the
streaming component as follows:


from("stream:file?fileName=src/data/webservices_20090723_001_trunc.log&groupLines=2")

Now this does group two lines together however it removes the new line
seperator. Consequently two records are concatenated in a single list entry
when the message arrives at doHandleCsvData.

I suppose I can write my own producer within the File component that takes
the file handle and streams data out. However I am keen on exploring the
capabilities of the existring components.

Would appreciate any help.

Thanks & Regards,

Anurag


--
View this message in context: http://camel.465427.n5.nabble.com/Grouping-lines-while-streaming-tp5497878p5497878.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Mime
View raw message