Return-Path: Delivered-To: apmail-camel-dev-archive@www.apache.org Received: (qmail 62480 invoked from network); 13 Jul 2009 16:21:40 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 13 Jul 2009 16:21:40 -0000 Received: (qmail 5464 invoked by uid 500); 13 Jul 2009 16:21:50 -0000 Delivered-To: apmail-camel-dev-archive@camel.apache.org Received: (qmail 5416 invoked by uid 500); 13 Jul 2009 16:21:50 -0000 Mailing-List: contact dev-help@camel.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@camel.apache.org Delivered-To: mailing list dev@camel.apache.org Received: (qmail 5406 invoked by uid 99); 13 Jul 2009 16:21:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Jul 2009 16:21:50 +0000 X-ASF-Spam-Status: No, hits=0.2 required=10.0 tests=SPF_PASS,URIBL_GREY,URIBL_RED X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of claus.ibsen@gmail.com designates 209.85.218.214 as permitted sender) Received: from [209.85.218.214] (HELO mail-bw0-f214.google.com) (209.85.218.214) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Jul 2009 16:21:39 +0000 Received: by bwz10 with SMTP id 10so2020624bwz.20 for ; Mon, 13 Jul 2009 09:21:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type :content-transfer-encoding; bh=3UUxRBU19X1XJg1dCfJ2pnGDNDhqrtpaNtKaqz62I18=; b=pSPKeNWdl64SraCTliEYWAWTAp1+wlYzoIoJLpeAHdJMxD3ITYYNAuXOEGV8bOMTdG QnmE67ERZuO8vqYO2BO4xSWEh68YOr8x0s+yt4/fXSQBJlQD6D4qiTieYVJ/h5bhWUVB kDmfG2I7no2B/D50h5aqzshVvAbZ0MHmTYtVU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=ZhlvgjOBPn7QYyCEj9kRXsQo0iisPhyZpdC+KqvoklQShAz/XyJO7OlRNJJDWO9RZv rdK83obneXG3I73wCBBKiZ30ydKS/qXhXZnT/3IT6yQfSlhYrLMTi4ttemXbNe/AMhc0 TEpYaLHOvTiNsgM5lE0v1EOQsl+DKMbhh+2ns= MIME-Version: 1.0 Received: by 10.223.107.135 with SMTP id b7mr2190668fap.30.1247502077084; Mon, 13 Jul 2009 09:21:17 -0700 (PDT) In-Reply-To: References: <5380c69c0907092205x22dbee68kf9184ad68fb1543e@mail.gmail.com> <5380c69c0907100407o37e78f68y21172265d73be66e@mail.gmail.com> <5380c69c0907100426q7e05f999rb66f6c54f8e98df9@mail.gmail.com> <08620754-CF9D-4FFE-8008-953FEF3E0370@gmail.com> <5380c69c0907120223i33d3b687l3df885c96cba5aab@mail.gmail.com> From: Claus Ibsen Date: Mon, 13 Jul 2009 18:20:57 +0200 Message-ID: <5380c69c0907130920t1c243aafwd3ce59ba844c570d@mail.gmail.com> Subject: Re: [DISCUSS] Semantics of IN and OUT (was: Faults and Exceptions in Camel) To: dev@camel.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org On Mon, Jul 13, 2009 at 5:50 PM, James Strachan w= rote: > I'm confused. So rather than calling Exchange.getIn() to get the IN, > you'd have to do Exchange.getUnitOfWork().somethingOrOther() to get > the last IN? I don't see that as any simpler; making one of the 2 > messages harder to find, just moves the confusion somewhere else? Having access to the original input in a route path is very rarely requested by end users. In fact I think its even more confused if getIN always returned the original input. (what should that be if a message is routed over (to seda/from seda) a seda endpoint?) So what I am saying is that we should have a model similar to Mule and Spring Integration =3D simpler and easier to understand and use. And also simpler and easier to use within the Camel framework itself - eg all the code that copies from IN to OUT and OUT to IN and whatnot. The last IN =3D the message itself. Given this route: from("direct:start").to("bean:foo").processRef("bar").transform(constant("B= ye World")).processRef("me"); And we send a "Hello World" message to direct:start. The message would be bean:foo getMessage =3D "Hello World bar getMessage =3D "Hello World me getMessage =3D "Bye World Now imagine we mutate the messages in bean foo public String reverse(String s) { // return the reversed string } bean:foo getMessage =3D "Hello World" bar getMessage =3D "dlroW olleH" me getMessage =3D "Bye World" No more IN vs OUT and other confusing bits. getMessage gets you the payload as it is. If you have looked as much Camel code as I have you would also flag that the IN OUT is leading to confusing and broken code. And trust me its broken, when you use the Exchange, but the BODY extractor of ProducerTemplate is fairly good at guessing and returning the right one from OUT or IN :) If its really that inportant to get hold of the original input we can add a getter to it on the Exchange. - getInputMessage() - getOriginalMessage() or what a good name would be. And on top of this we do not lose headers as we do now when people use a processor and do exchange.getOut().setBody(myBody); And then all the headers from IN is lost. > > 2009/7/12 Claus Ibsen : >> Hi >> >> Starting the kettle to brew me a grok of hot coffee, as its sunday >> morning and I guess I got the energy to toss in my 2$ in the bucket of >> ideas >> Now that we are into the open talks and that it seems we can open the >> box with the IN, OUT, FAULT again. >> >> I advocate for only one message on the implementation side. On the API >> side we could still have some notion of IN / OUT to keep the API >> migration / impact simpler. But at the end having one message API >> makes it simpler. There could be a boolean to test whether an output >> has been set or not. Just as if we have a test for fault. >> >> >> Proposed Exchange API >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> =A0Message getMessage() >> =A0void setMessage(Message msg); >> >> =A0boolean messageChanged(); >> =A0 -- false =3D message have newer been mutated, eg its still the same = input >> =A0 -- true =3D the message or message body have been mutated in some wa= y >> (setMessage or setBody called) >> =A0 -- need to find a good name for this method, eg testing whether the >> message have been changed/updated or not >> >> That leaves us with a simple API. >> >> Then we do not need to worry about all the message management >> internally in Camel, where we need to figure out whether to set data >> on IN or OUT and what else we kinda do, and it gets a bit complex over >> time how to do this correctly. >> >> >> MEP: InOnly >> =3D=3D=3D=3D=3D=3D=3D=3D=3D >> Now we can do as Hadrian want. Newer return something in OUT. And >> leave the original input on the Exchange. >> Here we need to use James CopyOnWrite technique so we can preserve the >> original message in case its mutated during routing. >> >> >> MEP: InOut >> =3D=3D=3D=3D=3D=3D=3D=3D=3D >> If an OUT message have been set (eg there is a reply), can be >> determined if the messageChanged() =3D=3D true. (then its not the origin= al >> message anymore). And yes the current code does this. It will just >> copy whatever there is in IN and use it as OUT if no OUT was set. >> With this proposal this will be improved as its easier to determine if >> there is an OUT message or not. >> >> >> Original message >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> Currently there is an API to get the original message from the >> UnitOfWork as its needed when doing redeliveries and a message was >> doomed >> and had to be moved to the dead letter channel. Then it makes much >> more sense to move the original message than the current mutated >> message. >> ( i wonder if it should be default behavior ) >> >> >> AggregationStrategy >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> Another benefit with a single getMessage() is that end users using >> AggregationStrategy will not be confused how to use it. >> Should I look in IN or OUT. And the current code can actually be a >> "lucky draw" as whether the data is in IN or OUT depending on facts >> such as how the route path is. >> We can even change it to pass in Message object instead of bare bone >> Exchange. You can always go from Message to exchange using >> getExchange(). >> >> >> All the processors / components / data formats >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> Logic will be easier as well as they do not need to cater for IN / OUT >> and where and how to set a result. Just work on the Message. >> >> >> Use @deprecated for steady migration >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D >> Hadrian suggested that for the API migration to a single getMessage >> you could let the getIn/setIn getOut/setOut delegate to >> getMessage/setMessage. >> And then mark them as @deprecated and then gradually change the camel >> code as we goes. So I do not think the hold up / change would take a >> lot of time and energy to get done. >> >> >> Final words >> =3D=3D=3D=3D=3D=3D=3D=3D >> So if it was possible to simplify the API and reduce the IN OUT FAULT >> to a simpler API, even go as far as I would like with a single >> Message. >> Then that would be really great and worth a hold up for a imminent 2.0 r= elease. >> >> >> Other frameworks >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> I guess it wasn't final words after all :) Just wanted to say that >> Mule, Spring Integration also just have a single message for the >> message. >> And yes I think their projects are also successful so it is not a loss >> that they do not have IN OUT. In fact I think their API is easier to >> work with than Camel. >> >> We are fortunate that most people with Camel do not work directly with >> Exchange but work more with Camel returning the message as an expected >> body type using its type converters. That helps a lot. >> >> But we have stories form Camel 1.x where people get confused whey >> Camel return their original input in some situations where it was not >> expected in the OUT message. Or the fact producer template sendBody is >> not void in 1.x. Then people think it can be used for InOut. >> >> Okay end of mail. >> >> >> On Fri, Jul 10, 2009 at 5:16 PM, James Strachan wrote: >>> 2009/7/10 Hadrian Zbarcea : >>>> Moved this slightly different topic to a separate thread. >>>> >>>> ++1 from me. >>>> >>>> Do we need a vote on this one or it's a consensus? =A0Claus, it looks = like you >>>> agree, but pointed out that there is some work involved, correct? >>>> >>>> Hadrian >>>> >>>> >>>> >>>> On Jul 10, 2009, at 8:40 AM, James Strachan wrote: >>>> >>>>> 2009/7/10 Claus Ibsen : >>>>>> >>>>>> On Fri, Jul 10, 2009 at 1:20 PM, James Strachan >>>>>> wrote: >>>>>>> >>>>>>> 2009/7/10 Claus Ibsen : >>>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> Another update on the IN vs OUT when you send in an Exchange. >>>>>>>> >>>>>>>> The ProducerCache that is doing the actual sending when using temp= late >>>>>>>> or sendTo etc, its basically doing all send X to endpoint. >>>>>>>> >>>>>>>> Well I am playing with to let it adhere to the principle Hadrian >>>>>>>> pointed out. He wanted the IN to be more static. >>>>>>>> And we cannot get there yet when you do routing as all the process= ors >>>>>>>> rely on IN being able to mutate during routing. >>>>>>>> >>>>>>>> Anyway my grief is that when you send in an Exchange the result wo= uld >>>>>>>> sometimes be stored on IN and not OUT. >>>>>>>> What I want it to do always is to store the result in OUT and keep= IN >>>>>>>> as the original input. >>>>>>>> >>>>>>>> The code to do this is now a bit more complex than just before >>>>>>>> >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 // copy the original input >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 Message original =3D exchange.getIn().= copy(); >>>>>>>> >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 producer.process(exchange); >>>>>>>> >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 // if no OUT then set current IN as re= sult (except for >>>>>>>> optional out) >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (!exchange.hasOut() && exchange.get= Pattern() !=3D >>>>>>>> ExchangePattern.InOptionalOut) { >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 // but only if its not the sam= e as original IN to >>>>>>>> avoid duplicating it >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 // and to adhere to the fact t= hat there was no OUT >>>>>>>> result at all >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (original.getBody() !=3D nu= ll && >>>>>>>> !original.getBody().equals(exchange.getIn().getBody())) { >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 exchange.setOut(exchan= ge.getIn()); >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 } >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 // and restore original in >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 exchange.setIn(original); >>>>>>>> >>>>>>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return exchange; >>>>>>>> >>>>>>>> >>>>>>>> What I need to do is to copy the original IN message as it can be >>>>>>>> mutated during routing. >>>>>>> >>>>>>> How about we prevent mutation of the IN message? Create a Message >>>>>>> facade which throws UnsupportedOperationException if you try to mut= ate >>>>>>> it in any way. Then we can pass the same read-only Message around a= s >>>>>>> the IN within retry loops or from step to step if no new output is >>>>>>> created (e.g. in a content based router where you just move a Messa= ge >>>>>>> to the right endpoint without changing it) >>>>>>> >>>>>> >>>>>> A good idea but will break a lot of logic in Camel. >>>>> >>>>> Agreed. But with the benefit that we'd be able to get rid of all the >>>>> defensive copies in our code; plus we'd be able to pass the same >>>>> Message from step to step. The API would be a bit more clean; to >>>>> change the output, you create an OUT message (maybe by copying the >>>>> IN). >>>>> >>>>> >>>>>> Most of the Camel >>>>>> processors work on the IN message and set the result on either IN or >>>>>> OUT. At best they set it on OUT. But then the IN is always the >>>>>> original input? Or am I mistaking? >>>>> >>>>> Yeah, we'd have to patch code to no longer mutate IN >>>>> >>>>> >>>>>> How will this work with the Pipes And Filters EIP if the IN is >>>>>> immutable and always the original input? >>>>> >>>>> If no OUT, then no output was created, so pass the IN along... >>>>> >>>>> >>>>> OK how about this; a CopyOnWriteMessageFacade which does not mutate >>>>> the original message at all ever; if a message is used in a purely >>>>> read only way, it does nothing but delegate to the original message - >>>>> but then as soon as someone mutates it, it creates a copy and uses >>>>> that from that point on? >>>>> >>>>> i.e. make the copy of the message lazy - and only make a copy when a >>>>> Processor really does try to mutate the Message? >>>>> >>>>> Then we'd get the best of both worlds; avoid breaking old code but >>>>> avoid tons of unnecessary copies? >>> >>> BTW then using the current API you could have a Message and then call >>> >>> Message origin =3D ...; >>> Message newMsg =3D origin.copy().copy().copy().copy(); >>> >>> and the message would not actually be copied at all; new would just be >>> a CopyOnWriteMessageFacade which would hold a reference to 'origin' as >>> the readOnlyMessage (which it never mutates). >>> >>> The copy would only take place if you did >>> >>> newMsg.setBody("foo") >>> >>> -- >>> James >>> ------- >>> http://macstrac.blogspot.com/ >>> >>> Open Source Integration >>> http://fusesource.com/ >>> >> >> >> >> -- >> Claus Ibsen >> Apache Camel Committer >> >> Open Source Integration: http://fusesource.com >> Blog: http://davsclaus.blogspot.com/ >> Twitter: http://twitter.com/davsclaus >> > > > > -- > James > ------- > http://macstrac.blogspot.com/ > > Open Source Integration > http://fusesource.com/ > --=20 Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus