Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
Received-SPF: neutral (athena.apache.org: 216.139.236.26 is neither permitted
 nor denied by domain of anurag.das.sharma@gmail.com)
Date: Sun, 19 Feb 2012 16:59:09 -0800 (PST)
From: Anurag Sharma <anurag.das.sharma@gmail.com>
To: users@camel.apache.org
Message-ID: <1329699549052-5497878.post@n5.nabble.com>
Subject: Grouping lines while streaming
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi All,

I have a file with million plus records. Each record is of CSV format. Now
instead of reading the whole file in memory I would like to stream it record
by record , or prefereably bunch N nbr of records in a single message
exchange.

Following is my route which works OK for streaming it line by line.


from("file:src/data?fileName=webservices_20090723_001_trunc.log&noop=true").split(body().tokenize("\n")).streaming()
        	  .to("seda:input?concurrentConsumers=1")
    	  .log("Processing ${id}");  	
    	from("seda:input?concurrentConsumers=1")
    	  .convertBodyTo(String.class)
    	  .unmarshal(csv)
  	  .to("bean:LogService?method=doHandleCsvData");

Now if i want to bunch a group of lines together, it appears that there is
no group option while streaming from file component. So I decided to try the
streaming component as follows:


from("stream:file?fileName=src/data/webservices_20090723_001_trunc.log&groupLines=2")

Now this does group two lines together however it removes the new line
seperator. Consequently two records are concatenated in a single list entry
when the message arrives at doHandleCsvData.

I suppose I can write my own producer within the File component that takes
the file handle and streams data out. However I am keen on exploring the
capabilities of the existring components.

Would appreciate any help.

Thanks & Regards,

Anurag


--
View this message in context: http://camel.465427.n5.nabble.com/Grouping-lines-while-streaming-tp5497878p5497878.html
Sent from the Camel - Users mailing list archive at Nabble.com.