camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Willem Jiang <willem.ji...@gmail.com>
Subject Re: Does stream based splitter support a property to indicate end of splitting?
Date Tue, 12 Jan 2010 05:11:59 GMT
I think your first solution is a more easy way to understand and you can 
do what you want in your customer bean. It also can handle the exception 
of splitting.

What you try to do is wrapping the file-to-db solution in on split tag, 
That is not we want camel to do.

Even we provide a interface to call the release strategy in Camel split, 
that will make your customer bean dependent on the Camel xxxStrategy 
Interface.

In your case, I think the split end flag is optional,  you 
saveToDatabase will break the loop if there is no further message, and 
you can finally release the resources those you use during the splitting.

Camel bean gives you the power to control your business logical, so 
don't wast it :)

Willem

ext2 wrote:
> Hi:
> 	I haven't create a JIRA yet, I thinks I should illustrate my
> question more clearly to see if the end-marker is worth to add.
> 
> 	My question could be illustrated by a simplified application:
> consume a very large file which store record , and save all the record of
> the file to database.
> 
> First: following is my first solution  using current camel: 
> <from uri="file://source?consumer.regexPattern="*\.dat"> 
> <doTry>
> <split stopOnException="true">
> 	<method bean="fileSplitBean" method="split"/>
> 	<bean ref="saveToDatabase">
> </split>
> <doFinally>
> 	<bean ref="fileSplitBean" method="release"/>
> </doFinally>
> </doTry>
> 
> The bean "fileSplitBean" has two method : method "split" used by
> split-pattern will open the file, and create a iterator to enumeration the
> record in file. Method "release" will close the opened file; 
> 
> But for me, what I feel uncomfortable is "normal process action"(here is
> fileSplitBean.split) and "release action"(here is fileSplitBean.release) is
> separated in two place. The sample I give is very simple, but if route is
> more complex, at just a glance, people cannot realized the fact that: the
> bean used for splitting also has a concerned "release action"; In another
> words, the solution is too close to the program-language.
> 
> Next , I tried camel to let the "normal-process action"and "release action"
> appeared more closely in the route. So the solution will appears as
> following: 1)change the iterator, and let the iterator return a very special
> end-flag message after all the record of file has been enumerated;2) the
> bean "saveToDatabase" knows the end-flag message,and do nothing for it. 3)
> give a aggregate for the splitter and the aggregator will test if the
> message is end-flag to see. But this solution look too odd, and even cannot
> works while exception occurs; 
> 
> So in order to enhance the solution , I thought about to let split-pattern
> give the end-flag, and using a aggregator to close file; the solution as
> following:
> <from uri="file://source?consumer.regexPattern="*\.dat"> 
> <split stopOnException="true" strategyRef="releaseStrategy">
> 	<method bean="fileSplitBean" method="split"/>
> 	<bean ref="saveToDatabase">
> </split>
> <bean id="releaseStrategy" class="...">
> 	<property name="fileSplitter" ref="fileSplitBean/>
> </bean>
> Class ReleaseStrategy{
> 	FileSplitBean fileSplitter;
> 	public Exchange aggregate(Exchange oldExchange, Exchange
> newExchange) {
> 	
> if(Boolean.True.equals(newExchange.getIn().getHeader(SPLIT_FINISHED)))
> 		fileSplitter.rease();
> 	}
> }
> 
> But the solution still have some-things unresolved:  1:cannot invoke release
> method while error occurs; 2:if split-pattern is nested, how to distinguish
> the end-flag.
> 
> Finally , what I really want is give the split-patter another strategy:
> release-strategy; but this solution maybe is too close to
> special-application. And I doesn't know if it worth to be adopted by camel.
> I illustrate it as following
> 
> <from uri="file://source?consumer.regexPattern="*\.dat"> 
> <split stopOnException="true" releaseStrategyRef="releaseStrategy">
> 	<method bean="fileSplitBean" method="split"/>
> 	<bean ref="saveToDatabase">
> </split>
> <bean id="releaseStrategy" class="...">
> 	<property name="fileSplitter" ref="fileSplitBean/>
> </bean>
> Class ReleaseStrategy{
> 	FileSplitBean fileSplitter;
> 	Public void release(Exchange exchange) {
> 		//to do check if exchange failed
> 		fileSplitter.release();
> 	}
> }
> 
> Willem Jiang wrote:
> +1 for this requirement. Please feel free to create a JIRA[1] for it.
> 
> [1] https://issues.apache.org/activemq/browse/CAMEL
> 
> Claus Ibsen wrote:
>> Hi
>>
>> No there is currently no such end marker, but I do think its possible
>> to add such one in the Splitter.
>>
>> Could you create a ticket and attach a small sample / unit test etc.
>> what you are doing?
>>
>>
>> On Mon, Jan 11, 2010 at 7:26 AM, ext2 <xuhb@tongtech.com> wrote:
>>> Hi:
>>>        By checking the camel's document, I just find a property tell the
>>> index of message, but cannot find a property to indicate the end of
>>> splitting.
>>>        But some-times, it will be very use-full for stream-based
> splitter.
>>> For example:
>>>        When I using stream-based splitter to process all the records
> stored
>>> in a large file sequence-ly. I will use a bean which return a iterator
> which
>>> read record sequence from the file(which store a lot of record ).After
>>> processed all the record, I must close the file. So I need a flag to
>>> indicate the end of  record and close the file;
>>>
>>>        If without such a property to indicate end of splitting, I must
>>> write a very special iterator which give a "End-Flag" message to say :
> "the
>>> record is all over". But the program of iterator looks very odd;
>>>
>>>        Does any-one knows about this?  Or a better suggest to resolve
>>> my-question?
>>>
>>> Thanks for any-suggestion;
>>>
>>>
>>>
>>>
>>
>>
> 
> 
> 
> 


Mime
View raw message