camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel)
Date Mon, 13 Jul 2009 16:20:57 GMT
On Mon, Jul 13, 2009 at 5:50 PM, James Strachan<james.strachan@gmail.com> wrote:
> I'm confused. So rather than calling Exchange.getIn() to get the IN,
> you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get
> the last IN? I don't see that as any simpler; making one of the 2
> messages harder to find, just moves the confusion somewhere else?

Having access to the original input in a route path is very rarely
requested by end users.
In fact I think its even more confused if getIN always returned the
original input. (what should that be if a message is routed over (to
seda/from seda) a seda endpoint?)

So what I am saying is that we should have a model similar to Mule and
Spring Integration = simpler and easier to understand and use.
And also simpler and easier to use within the Camel framework itself -
eg all the code that copies from IN to OUT and OUT to IN and whatnot.

The last IN = the message itself.

Given this route:
from("direct:start").to("bean:foo").processRef("bar").transform(constant("Bye
World")).processRef("me");

And we send a "Hello World" message to direct:start. The message would be

bean:foo
getMessage = "Hello World

bar
getMessage = "Hello World

me
getMessage = "Bye World


Now imagine we mutate the messages in bean foo
public String reverse(String s) {
   // return the reversed string
}


bean:foo
getMessage = "Hello World"

bar
getMessage = "dlroW olleH"

me
getMessage = "Bye World"


No more IN vs OUT and other confusing bits. getMessage gets you the
payload as it is.

If you have looked as much Camel code as I have you would also flag
that the IN OUT is leading to confusing and broken code.
And trust me its broken, when you use the Exchange, but the BODY
extractor of ProducerTemplate is fairly good at guessing and returning
the right one from OUT or IN :)


If its really that inportant to get hold of the original input we can
add a getter to it on the Exchange.
- getInputMessage()
- getOriginalMessage()
or what a good name would be.


And on top of this we do not lose headers as we do now when people use
a processor and do
exchange.getOut().setBody(myBody);

And then all the headers from IN is lost.



>
> 2009/7/12 Claus Ibsen <claus.ibsen@gmail.com>:
>> Hi
>>
>> Starting the kettle to brew me a grok of hot coffee, as its sunday
>> morning and I guess I got the energy to toss in my 2$ in the bucket of
>> ideas
>> Now that we are into the open talks and that it seems we can open the
>> box with the IN, OUT, FAULT again.
>>
>> I advocate for only one message on the implementation side. On the API
>> side we could still have some notion of IN / OUT to keep the API
>> migration / impact simpler. But at the end having one message API
>> makes it simpler. There could be a boolean to test whether an output
>> has been set or not. Just as if we have a test for fault.
>>
>>
>> Proposed Exchange API
>> ==================
>>
>>  Message getMessage()
>>  void setMessage(Message msg);
>>
>>  boolean messageChanged();
>>   -- false = message have newer been mutated, eg its still the same input
>>   -- true = the message or message body have been mutated in some way
>> (setMessage or setBody called)
>>   -- need to find a good name for this method, eg testing whether the
>> message have been changed/updated or not
>>
>> That leaves us with a simple API.
>>
>> Then we do not need to worry about all the message management
>> internally in Camel, where we need to figure out whether to set data
>> on IN or OUT and what else we kinda do, and it gets a bit complex over
>> time how to do this correctly.
>>
>>
>> MEP: InOnly
>> =========
>> Now we can do as Hadrian want. Newer return something in OUT. And
>> leave the original input on the Exchange.
>> Here we need to use James CopyOnWrite technique so we can preserve the
>> original message in case its mutated during routing.
>>
>>
>> MEP: InOut
>> =========
>> If an OUT message have been set (eg there is a reply), can be
>> determined if the messageChanged() == true. (then its not the original
>> message anymore). And yes the current code does this. It will just
>> copy whatever there is in IN and use it as OUT if no OUT was set.
>> With this proposal this will be improved as its easier to determine if
>> there is an OUT message or not.
>>
>>
>> Original message
>> =============
>> Currently there is an API to get the original message from the
>> UnitOfWork as its needed when doing redeliveries and a message was
>> doomed
>> and had to be moved to the dead letter channel. Then it makes much
>> more sense to move the original message than the current mutated
>> message.
>> ( i wonder if it should be default behavior )
>>
>>
>> AggregationStrategy
>> ===============
>> Another benefit with a single getMessage() is that end users using
>> AggregationStrategy will not be confused how to use it.
>> Should I look in IN or OUT. And the current code can actually be a
>> "lucky draw" as whether the data is in IN or OUT depending on facts
>> such as how the route path is.
>> We can even change it to pass in Message object instead of bare bone
>> Exchange. You can always go from Message to exchange using
>> getExchange().
>>
>>
>> All the processors / components / data formats
>> ===================================
>> Logic will be easier as well as they do not need to cater for IN / OUT
>> and where and how to set a result. Just work on the Message.
>>
>>
>> Use @deprecated for steady migration
>> ============================
>> Hadrian suggested that for the API migration to a single getMessage
>> you could let the getIn/setIn getOut/setOut delegate to
>> getMessage/setMessage.
>> And then mark them as @deprecated and then gradually change the camel
>> code as we goes. So I do not think the hold up / change would take a
>> lot of time and energy to get done.
>>
>>
>> Final words
>> ========
>> So if it was possible to simplify the API and reduce the IN OUT FAULT
>> to a simpler API, even go as far as I would like with a single
>> Message.
>> Then that would be really great and worth a hold up for a imminent 2.0 release.
>>
>>
>> Other frameworks
>> =============
>> I guess it wasn't final words after all :) Just wanted to say that
>> Mule, Spring Integration also just have a single message for the
>> message.
>> And yes I think their projects are also successful so it is not a loss
>> that they do not have IN OUT. In fact I think their API is easier to
>> work with than Camel.
>>
>> We are fortunate that most people with Camel do not work directly with
>> Exchange but work more with Camel returning the message as an expected
>> body type using its type converters. That helps a lot.
>>
>> But we have stories form Camel 1.x where people get confused whey
>> Camel return their original input in some situations where it was not
>> expected in the OUT message. Or the fact producer template sendBody is
>> not void in 1.x. Then people think it can be used for InOut.
>>
>> Okay end of mail.
>>
>>
>> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan<james.strachan@gmail.com> wrote:
>>> 2009/7/10 Hadrian Zbarcea <hzbarcea@gmail.com>:
>>>> Moved this slightly different topic to a separate thread.
>>>>
>>>> ++1 from me.
>>>>
>>>> Do we need a vote on this one or it's a consensus?  Claus, it looks like
you
>>>> agree, but pointed out that there is some work involved, correct?
>>>>
>>>> Hadrian
>>>>
>>>>
>>>>
>>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote:
>>>>
>>>>> 2009/7/10 Claus Ibsen <claus.ibsen@gmail.com>:
>>>>>>
>>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan<james.strachan@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> 2009/7/10 Claus Ibsen <claus.ibsen@gmail.com>:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Another update on the IN vs OUT when you send in an Exchange.
>>>>>>>>
>>>>>>>> The ProducerCache that is doing the actual sending when using
template
>>>>>>>> or sendTo etc, its basically doing all send X to endpoint.
>>>>>>>>
>>>>>>>> Well I am playing with to let it adhere to the principle
Hadrian
>>>>>>>> pointed out. He wanted the IN to be more static.
>>>>>>>> And we cannot get there yet when you do routing as all the
processors
>>>>>>>> rely on IN being able to mutate during routing.
>>>>>>>>
>>>>>>>> Anyway my grief is that when you send in an Exchange the
result would
>>>>>>>> sometimes be stored on IN and not OUT.
>>>>>>>> What I want it to do always is to store the result in OUT
and keep IN
>>>>>>>> as the original input.
>>>>>>>>
>>>>>>>> The code to do this is now a bit more complex than just before
>>>>>>>>
>>>>>>>>               // copy the original input
>>>>>>>>               Message original = exchange.getIn().copy();
>>>>>>>>
>>>>>>>>               producer.process(exchange);
>>>>>>>>
>>>>>>>>               // if no OUT then set current IN as
result (except for
>>>>>>>> optional out)
>>>>>>>>               if (!exchange.hasOut() && exchange.getPattern()
!=
>>>>>>>> ExchangePattern.InOptionalOut) {
>>>>>>>>                   // but only if its not the same
as original IN to
>>>>>>>> avoid duplicating it
>>>>>>>>                   // and to adhere to the fact that
there was no OUT
>>>>>>>> result at all
>>>>>>>>                   if (original.getBody() != null
&&
>>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) {
>>>>>>>>                       exchange.setOut(exchange.getIn());
>>>>>>>>                   }
>>>>>>>>               }
>>>>>>>>               // and restore original in
>>>>>>>>               exchange.setIn(original);
>>>>>>>>
>>>>>>>>               return exchange;
>>>>>>>>
>>>>>>>>
>>>>>>>> What I need to do is to copy the original IN message as it
can be
>>>>>>>> mutated during routing.
>>>>>>>
>>>>>>> How about we prevent mutation of the IN message? Create a Message
>>>>>>> facade which throws UnsupportedOperationException if you try
to mutate
>>>>>>> it in any way. Then we can pass the same read-only Message around
as
>>>>>>> the IN within retry loops or from step to step if no new output
is
>>>>>>> created (e.g. in a content based router where you just move a
Message
>>>>>>> to the right endpoint without changing it)
>>>>>>>
>>>>>>
>>>>>> A good idea but will break a lot of logic in Camel.
>>>>>
>>>>> Agreed. But with the benefit that we'd be able to get rid of all the
>>>>> defensive copies in our code; plus we'd be able to pass the same
>>>>> Message from step to step. The API would be a bit more clean; to
>>>>> change the output, you create an OUT message (maybe by copying the
>>>>> IN).
>>>>>
>>>>>
>>>>>> Most of the Camel
>>>>>> processors work on the IN message and set the result on either IN
or
>>>>>> OUT. At best they set it on OUT. But then the IN is always the
>>>>>> original input? Or am I mistaking?
>>>>>
>>>>> Yeah, we'd have to patch code to no longer mutate IN
>>>>>
>>>>>
>>>>>> How will this work with the Pipes And Filters EIP if the IN is
>>>>>> immutable and always the original input?
>>>>>
>>>>> If no OUT, then no output was created, so pass the IN along...
>>>>>
>>>>>
>>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate
>>>>> the original message at all ever; if a message is used in a purely
>>>>> read only way, it does nothing but delegate to the original message -
>>>>> but then as soon as someone mutates it, it creates a copy and uses
>>>>> that from that point on?
>>>>>
>>>>> i.e. make the copy of the message lazy - and only make a copy when a
>>>>> Processor really does try to mutate the Message?
>>>>>
>>>>> Then we'd get the best of both worlds; avoid breaking old code but
>>>>> avoid tons of unnecessary copies?
>>>
>>> BTW then using the current API you could have a Message and then call
>>>
>>> Message origin = ...;
>>> Message newMsg = origin.copy().copy().copy().copy();
>>>
>>> and the message would not actually be copied at all; new would just be
>>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as
>>> the readOnlyMessage (which it never mutates).
>>>
>>> The copy would only take place if you did
>>>
>>> newMsg.setBody("foo")
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://fusesource.com/
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message