camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hadrian Zbarcea <hzbar...@gmail.com>
Subject Re: Camel 2.0 - FAULT message, do we need it?
Date Tue, 23 Sep 2008 13:40:30 GMT
Hi Vadim,

I am glad we are on the same page now.

That's the thing, I don't think we spent enough time to nail down all  
the things your asking.  We may not have the right answer or an answer  
at all (yet) for what a well behaved component should do with respect  
to faults.  Afaik, even now, the sql component you refer to does not  
distinguish between transient and persistent faults, nor the http  
component.  So the overloading of the server does take place already.

We do have a mechanism in place today to force treating faults as  
exceptions, but for a less than trivial route I am not sure how  
practical that is.  As an example, the same fault thrown by different  
endpoints in the route may have to be handled differently, and I am  
not sure enough information is provided right now to make that  
possible (maybe a new header with the url of the endpoint throwing the  
fault?).

Let's use camel-316 and add all ideas there of what a well behaved  
component should do with respect to faults.

Thanks for your comments and help,
Hadrian


On Sep 22, 2008, at 11:37 PM, Vadim Chekan wrote:

> Hi Hadrian,
> Thanks for explaining, I understand it now what you mean.
>
> About terminology. May be we'd better stick with "transient" and  
> "persistent" because with "faults" and "exceptions" I'll have to  
> look it up email each time to remember which one is it.
>
> I totally agree with you on having this distinguish between two  
> types of  faults (I hope I got it right this time :). Many reties of  
> inserting large batch into SQL when there is foreign key violation  
> can overload sql server.
>
> As I understand it, components will be responsible for deciding what  
> is transient and what is not. For example jdbc component can  
> interpret the same SQL exception class differently. If it is a  
> timeout or a deadlock, there is sense to try it again. But if it is  
> a key violation or wrong login then it is persistent.
>
> So what are (will be) the rules for well behaved component? Perhaps  
> we want to keep it backward compatible and in case if exception is  
> not handled by component it is interpreted as transient error?
> So if component decides that error is persistent then what? Set  
> "fault" of exchange to an exception? Or there will be a special  
> property in runtime exception?
>
> Vadim.
>
> Hadrian Zbarcea wrote:
>> I reluctantly pick the gauntlet.  I feel your passion, and it looks  
>> to me more like violent agreement than disagreement (at least of  
>> the fact that there is some cleanup to be done).  So let's clarify  
>> a few things.
>> I am not sure how carefully you read https://issues.apache.org/activemq/browse/CAMEL-316

>>  and the thread referenced there.  I think the consensus is to  
>> combine faults and exceptions, we just have to nail out the details.
>>> So distinguishing fault and exception in Camel is senseless. There  
>>> is no difference between two.
>> I certainly understand this (your) point of view.  I wish things  
>> were as clear cut as you suggest, but I am not convinced that's the  
>> case, on the contrary.
>> I'll move away from the terms fault and exception for a bit.  There  
>> are two kinds of errors: transient and persistent.  Transient  
>> errors indicate some environmental problem that prevented carrying  
>> out the request.  Persistent errors indicate something wrong with  
>> the request.  In the former case, if the caller were to issue the  
>> request again at a later time, it may get it processed  
>> successfully.  In the latter case, no amount of retries would help.
>> Some protocols/technologies distinguish between the two kinds of  
>> errors, some do not and some are in between (think http:, on a 500  
>> you might want to retry, on a 404, 403, 401 one shouldn't!).  Wsdl  
>> for instance provides the faults.  Some (architects) use them to  
>> model the persistent errors a system may return, some do not.  What  
>> is the intended audience for faults?  The caller.  From this point  
>> of view yes, camel, which is part of the infrastructure should not  
>> care about faults, they are an output like any other, which happens  
>> to be an exception, which should be sent back to the caller to take  
>> appropriate action.  You might say that from a Camel perspective a  
>> Fault is *not* an Exception, it's a valid reply to be sent back to  
>> the caller.
>> You ask "What "not in the way intended" mean?", well it means that  
>> on a financial transfer for instance, you may get an "insufficient  
>> funds" fault that indicates that the server got the request and for  
>> the reason provided decided not to carry out the request in the  
>> intended way, i.e. to actually complete the transfer and no amount  
>> of retries would change that until somebody opens the purse and  
>> deposits more funds in the said account.  More precisely for me  
>> "the way intended" means such as to get a valid response/output,  
>> not a fault.  That is more often than not, when requests are sent,  
>> it is done with the intention of receiving a valid output, not a  
>> fault (which may not be true for a test application, but I think  
>> you got the point, sorry for not being more clear in my past post).
>> Good infrastructure relieves the caller as much as possible of  
>> dealing with transient errors taking the appropriate measures  
>> (retries, whatever), we have the ErrorHandler for that.  Great  
>> infrastructure allows one to deal with (at least some of the)  
>> persistent errors.  Why?  Because in many cases they way to handle  
>> them is the same, so that could be automated.  So in that case you  
>> could see the Fault as *being* an Exception and the infrastructure  
>> could use predefined rules to handle them.
>> The issue is not if a Fault is of a java.lang.Exception type (I  
>> think there was a comment stating that: of course Faults are  
>> Exceptions, meaning of java.lang.Exception type).  It may very well  
>> be.  From a web services framework point of view, they can be  
>> modeled as of java.lang.Exception type.  But Camel is a router, the  
>> question is if and how should Camel handle Faults.  My point is  
>> that I would prefer to not lose the semantical distinction between  
>> fault and exception, always react to exceptions and optionally  
>> react to fault (and we do some of that).
>> As far as implementation goes, I could see faults going as outputs  
>> (and being of java.lang.Exception type) so one can have rules to  
>> handle them *if* desired.  Or we can a header saying that this  
>> output is actually a fault, so handling could be based on that.   
>> (That means that in my mind a Fault is more of an output than an  
>> Exception in Camel Exchange sense).
>> To your second point about In/Out.  The In is always read-only.   
>> One never modifies the in, this is always the message that was sent  
>> by the producer.  It's history, doesn't change.  Out is always the  
>> response.  And there is the exchange pattern: InOnly, the Out is  
>> always null, InOut, then well the out is null.  There is that  
>> ambiguity by convention that the next in is either last out of *if  
>> null* the last in.  This should be clarified indeed and produced  
>> confusion more than once.
>> In your filter example, yeah, of course the result is a message,  
>> but think of the exchange as having the whole context, the message  
>> before the filter, the one after, the exception (if any) and some  
>> context/properties/headers (maybe).  I am not sure if a null output  
>> should mean "stop" the pipeline.  For an InOut exchange patter,  
>> maybe, although I doubt this happens a lot in practice.  For an  
>> InOnly pattern (such as log, wiretap, etc, that don't have an  
>> output, they're not an 'echo') I see nothing wrong with doing what  
>> we already do, which is sending the same In to the next processor.   
>> Keep in mind that some technologies do not have a way to formalize  
>> the exchange pattern (such as wsdl does for instance) so the only  
>> way to distinguish between an InOnly and InOut is based on  
>> convention, such as the Out being null.
>> I wholeheartedly agree that this (more or less) edge cases should  
>> be properly clarified and documented, and I hope you'll keep the  
>> ball rolling, so we could hopefully get it done in 1.5.  I don't  
>> think it's too hard to do, but one, we need to agree on how to do  
>> it and two, minimize the impact of the api change.  (I think it's  
>> reasonable tough to keep the api, deprecate and never use the fault  
>> for 1.5).
>> Thoughts?
>> Hadrian
>> On Sep 14, 2008, at 5:23 PM, Vadim Chekan wrote:
>>> I even didn't know about FAULT :(
>>> Does it mean that all my processors are buggy because I do not  
>>> check fault?
>>>
>>> My understanding of Fault in SOAP is that *it is* an exception. WS  
>>> frameworks generate proxy classes which check fault and throw  
>>> exception. The reason Soap use fault is that it is the only way  
>>> you can serialize an exception.
>>> The same approach on server side: any business exception is caught  
>>> by WS handler, wrapped into fault code and sent back.
>>> So distinguishing fault and exception in Camel is senseless. There  
>>> is no difference between two.
>>> Fault code is nothing else but serialized exception.
>>>
>>> I'm afraid I can't agree with Hadrian:
>>>
>>> > Outputs provide a response after
>>> > successful processing.  Faults are responses that indicate to the
>>> > client that the expected goal was not achieved, and they usually
>>> > provide a reason.  Faults are messages that tell a client that the
>>> > server did receive the request and successfully processed it,  
>>> but not
>>> > in the way intended and the client may need to take further action
>>>
>>> What "not in the way intended" mean? What should Camel do if  
>>> something went not in the way intended? The answer is: nothing.  
>>> Camel knows nothing about application business stuff. From Camel  
>>> perspective message is either processed or not. Period. It is  
>>> application which should know that certain component can return  
>>> two types of responses and act accordingly. From Camel perspective  
>>> it is a perfectly valid response which is indistinguishable from  
>>> Out.
>>>
>>> Now, lets talk use cases. Lets say we have a batch to process.  
>>> Like any programmer I'm all about ACID so I design my system to  
>>> rollback and fail the whole batch if any item failed.
>>> But my users are unhappy. They come to me and say that if I just  
>>> create a list of errors and keep batch executing they would rather  
>>> fix errors manually instead of cleaning source data and re-running  
>>> the batch.
>>> So now I'm facing dilemma. If before knew that if no exception  
>>> happen then batch succeeded but now I have a "failed" response  
>>> with list of failed items. So I can not throw exception anymore. I  
>>> need to process this "failed" list by publishing it.
>>> But it is very important to understand that from service design  
>>> point of view there was no errors! I got a response with a list of  
>>> items which need to be published. Sounds like typical workflow. It  
>>> is error only from customer perspective, not from the Camel or  
>>> even application perspective!
>>>
>>> I provided this long example because I suspect that this "fault"  
>>> conception is interpretation of application-level faults. I want  
>>> to show that there exist no "fault" message as opposite to  
>>> exception. If you have a use case which demonstrates "fault"  
>>> message, lets discuss.
>>>
>>> In/Out
>>> ==============
>>> I must admit I had some confusion grasping "exchange" conception.  
>>> The word exchange assumes that there is 2-directional flow between  
>>> 2 entities. But Filter has only one direction: from one component  
>>> to another. So exchange between what and what is happening?
>>> In/Out is perspective of component. The component is given In  
>>> message and must provide the Out message. But it is not "exchange"  
>>> it is rather replacing In with Out.
>>> Now, on practical side, there is no need to provide both In and  
>>> Out at the same time. We can assume that component will return its  
>>> answer in the same object. But I'm failing to see any practical  
>>> benefits. IMHO it is the same, just matter of variable allocation.
>>> I think Roman's (and mine) frustration comes not from the fact  
>>> that there are in/out but from the fact that they are too relaxed  
>>> and it is not clear when In is Out :) So stricter rules should  
>>> address this concern.
>>>
>>> I have implemented  a small message routing system for internal  
>>> needs at works and I used Soap idea of Envelope with properties  
>>> and payload - body. So I have no Exchange entity as such. All  
>>> service information, like number of faults, etc is in the envelope  
>>> header. The processor function itself is an exchange:
>>> "public MessageEnvelope Process(MessageEnvelope env);"
>>> Later I faced a need to implement a splitter which produces more  
>>> then one message so I modified interface to return array of  
>>> messages:
>>> "public MessageEnvelope[] Process(MessageEnvelope env);"
>>> In Camel there is a special type of processor to handle this. In  
>>> my design any component can do broadcast by returning multiple  
>>> messages.
>>>
>>> You can do all the things you do in camel in/out exchange.
>>> You can return null which would mean "no outputs, stop  
>>> processing"; you can return exact the same object you received for  
>>> logging for example; you can create a copy of object, modify it  
>>> and return; you can create a bunch of new messages and return then.
>>>
>>> In/Out notation is pretty much the same as described above but  
>>> ability to modify In message and leave Out null brings  
>>> uncertainty. Should I modify In or create Out? Why the heck In  
>>> work as Out? Does componentX use In or Out? etc.
>>> I would think about making rules more strict: result is always in  
>>> Out. Providing many ways of doing the same thing without benefits  
>>> is a bad idea. You'll spend infinite amount of time explaining  
>>> newbies what in/out thing is. Principle of the least surprise :)
>>>
>>>
>>> Another note about HTTP request/response.
>>> I think it has nothing to do with Camel. Camel In/Out implements  
>>> conception of filter. You have a message asking to convert X $USD  
>>> to EUR. Your component use http to do it. So what the output of  
>>> your component is, http response? I doubt it. It is a message with  
>>> EUR amount. The whole http request-response is internal business  
>>> of component. Http request-response in an Exchange, when Camel  
>>> Filter is not.
>>>
>>> So my opinion is:
>>> 1. Faults must be removed and exceptions used instead like any WS  
>>> framework does.
>>> 2. "In" must be In and Out must be Out, meaning In can not be  
>>> modified and Out *must* be set; no tricks "if out is null then  
>>> interpret In as Out". We should agree what to do with "null" out.  
>>> Is it an error or it means "no messages to process, stop the pipe"?
>>>
>>> Vadim.
>>>
>>> P.S. Great discussion btw.
>>>
>>> Claus Ibsen wrote:
>>>> Hi
>>>> Just a quick mail to get the discussion rolling.
>>>> I think we should @deprecated exchange.fault in Camel 1.5 and  
>>>> remove it in Camel 2.0.
>>>> It is a confusing concept that isn't used much, and many  
>>>> components doesn't handle the FAULT message. For instance it was  
>>>> missing in camel-jms a very central component.
>>>> Med venlig hilsen
>>>> Claus Ibsen
>>>> ......................................
>>>> Silverbullet
>>>> Skovsgårdsvænget 21
>>>> 8362 Hørning
>>>> Tlf. +45 2962 7576
>>>> Web: www.silverbullet.dk
>>>
>


Mime
View raw message