camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem Jiang <willem.ji...@gmail.com>
Subject Re: Grouping lines while streaming
Date Mon, 20 Feb 2012 12:04:54 GMT
You can also create a customer processor[1] to combine the list of 
object into a String like this

from("stream:file?fileName=src/data/webservices_20090723_001_trunc.log&groupLines=2")
.processor(new Processor() {
     void process(Exchange exchange) {
        Message in = exchange.getIn();
        List<Object> lines = in.getBody();
        StringBuffer buffer = new StringBuffer();
        for(Object line:lines) {
              buffer.append(line);
              buffer.append("\n");
        }
        exchange.getOut().setBody(buffer);
     }
}).

On Mon Feb 20 15:55:07 2012, Willem Jiang wrote:
> It could more easy for user to group the lines if camel-stream 
> component can expose the strategy processLine to the client.
> I just fill a JIRA[1] for it
>
> You may need to extends the StreamConsumer yourself to implement such 
> of feature at this time.
>
> [1]https://issues.apache.org/jira/browse/CAMEL-5017
>
> On Mon Feb 20 08:59:09 2012, Anurag Sharma wrote:
>> Hi All,
>>
>> I have a file with million plus records. Each record is of CSV 
>> format. Now
>> instead of reading the whole file in memory I would like to stream it 
>> record
>> by record , or prefereably bunch N nbr of records in a single message
>> exchange.
>>
>> Following is my route which works OK for streaming it line by line.
>>
>>
>> from("file:src/data?fileName=webservices_20090723_001_trunc.log&noop=true").split(body().tokenize("\n")).streaming()

>>
>> .to("seda:input?concurrentConsumers=1")
>> .log("Processing ${id}");
>> from("seda:input?concurrentConsumers=1")
>> .convertBodyTo(String.class)
>> .unmarshal(csv)
>> .to("bean:LogService?method=doHandleCsvData");
>>
>> Now if i want to bunch a group of lines together, it appears that 
>> there is
>> no group option while streaming from file component. So I decided to 
>> try the
>> streaming component as follows:
>>
>>
>> from("stream:file?fileName=src/data/webservices_20090723_001_trunc.log&groupLines=2")

>>
>>
>> Now this does group two lines together however it removes the new line
>> seperator. Consequently two records are concatenated in a single list 
>> entry
>> when the message arrives at doHandleCsvData.
>>
>> I suppose I can write my own producer within the File component that 
>> takes
>> the file handle and streams data out. However I am keen on exploring the
>> capabilities of the existring components.
>>
>> Would appreciate any help.
>>
>> Thanks& Regards,
>>
>> Anurag
>>
>>
>> -- 
>> View this message in context: 
>> http://camel.465427.n5.nabble.com/Grouping-lines-while-streaming-tp5497878p5497878.html

>>
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>
>



-- 
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog:    http://willemjiang.blogspot.com (English)
         http://jnn.javaeye.com (Chinese)
Twitter: willemjiang 
Weibo: willemjiang 


Mime
View raw message