synapse-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Asankha C. Perera" <asan...@wso2.com>
Subject Re: Proposal to implement ws-eventing and an event distribution model in Synapse
Date Mon, 15 Sep 2008 11:25:11 GMT
I reverted by commit.. but still see failures:

Tests in error:
  
0001:test=AsyncXML,data=ASCII,messageType=SOAP11,client=java.net,endpoint=axis(org.apache.synapse.transport.testkit.tests.async.XMLAsyncMessageTestCase)
  
0034:test=AsyncTextPlain,data=ASCII,client=java.net,endpoint=axis(org.apache.synapse.transport.testkit.tests.async.TextPlainTestCase)
  
0035:test=AsyncTextPlain,data=UTF8,client=java.net,endpoint=axis(org.apache.synapse.transport.testkit.tests.async.TextPlainTestCase)
  
0036:test=AsyncTextPlain,data=Latin1,client=java.net,endpoint=axis(org.apache.synapse.transport.testkit.tests.async.TextPlainTestCase)
  
0043:test=AsyncBinary,client=java.net,endpoint=axis(org.apache.synapse.transport.testkit.tests.async.BinaryTestCase)
  
0046:test=REST,client=java.net,endpoint=axis(org.apache.synapse.transport.testkit.tests.async.RESTTestCase)
  
0143:test=EchoXML,data=ASCII,messageType=POX,broker=qpid,replyDestType=queue,destType=queue,contentTypeMode=TRANSPORT,client=axis,endpoint=mock(org.apache.synapse.transport.testkit.tests.echo.XMLRequestResponseMessageTestCase)

Andreas, can you fix these and checkin the changes

asankha


Asankha C. Perera wrote:
> Ruwan
>
> I am reverting my changes as we speak, until such time I can fix them 
> and make sure the tests run.. sorry for this..
>
> asankha
>
> Andreas Veithen wrote:
>> For the test failures: On Friday, Asankha made a change in the mail 
>> transport that causes 12 unit tests to fail. In addition to that, 
>> some tests in the transports module may fail on some 
>> platforms/systems because of concurrency issues. They run fine on my 
>> machines (Mac OS X and Windows XP), but Asankha told me that he sees 
>> this kind of failures (should be corrected in the meantime).
>>
>> Andreas
>>
>> Quoting Ruwan Linton <ruwan.linton@gmail.com>:
>>
>>> Sorry Paul, I was also unable to comment on your last queries.... as 
>>> you may
>>> know I am traveling. I think the above 2 points that you made have some
>>> interesting value in it. Therefore I would like to keep the ability to
>>> configure the event source and the notification service sections in the
>>> configuration.
>>>
>>> As you told, I just started the implementation and you can expect the
>>> initial version soon. For that lets go with the proposed 
>>> configuration and I
>>> will keep the two elements to configure the notification section and
>>> publishing section.
>>>
>>> BTW: I am seeing a test failure in the trunk? Do any one know the 
>>> reason for
>>> this? Andreas?
>>>
>>> (PS: please make sure all tests are passing before commit)
>>>
>>> Thanks,
>>> Ruwan
>>>
>>>
>>> On Mon, Sep 15, 2008 at 4:02 PM, Paul Fremantle <pzfreo@gmail.com> 
>>> wrote:
>>>
>>>> Any other comments on this? I know Ruwan is working on it and I know
>>>> we would appreciate feedback and thoughts.
>>>>
>>>> Paul
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Paul Fremantle <pzfreo@gmail.com>
>>>> Date: Fri, Sep 12, 2008 at 11:56 AM
>>>> Subject: Re: Proposal to implement ws-eventing and an event
>>>> distribution model in Synapse
>>>> To: dev@synapse.apache.org
>>>>
>>>>
>>>> Ruwan
>>>>
>>>> The reasons I think we should keep the WSEventing section are:
>>>>
>>>> 1) We should support other models - WS-Notification, etc. Although we
>>>> have started from Eventing, this is a fairly generic model of events
>>>> and I think we should keep it layered.
>>>>
>>>> 2) You may need to configure security or other aspects onto the
>>>> WSEventing endpoint, so you need to offer the same configuration
>>>> elements that proxy does (except target).
>>>>
>>>> Paul
>>>>
>>>> On Fri, Sep 12, 2008 at 10:01 AM, Ruwan Linton 
>>>> <ruwan.linton@gmail.com>
>>>> wrote:
>>>> > Paul,
>>>> >
>>>> > Very nice explanation of the concepts that we have been trying to 
>>>> put
>>>> > together into the code. Let me add some more to your explanation and
>>>> refine
>>>> > the configuration a bit more.
>>>> >
>>>> > <eventSource name="blah">
>>>> >     <subscriptionManager
>>>> > 
>>>> class="org.apache.synapse.events.DefaultInMemorySubscriptionManager">
>>>> >         <property name="blah">some xml prop</property>
>>>> >         <property name="other" value="some text property"/>
>>>> >     </subscriptionManager>
>>>> >     <staticSubscriptions>
>>>> >         <subscription id="static1">
>>>> >             <filter..../>
>>>> >             <sequence.../>
>>>> >             <endpoint../>
>>>> >         </subscription>*
>>>> >     <staticSubscriptions>?
>>>> > <eventSource/>
>>>> >
>>>> > Here I am getting rid of the wsEventing configuration element 
>>>> where you
>>>> > specify the subscription service and the event source service. So 
>>>> my idea
>>>> is
>>>> > we can extend the proxy services model here and create a new
>>>> > EventingMessageReceiver, which listens for all the requests 
>>>> coming to
>>>> this
>>>> > event source. (I must also say at this point event source is now a
>>>> service
>>>> > inside synapse and that fits with the model of extending the proxy
>>>> service
>>>> > behavior)
>>>> >
>>>> > This EventingMessageReceiver knows how to filter out the the 
>>>> subscription
>>>> > messages from the notification messages and it uses the specified
>>>> > subscription manager if it is a subscription request, and if it is a
>>>> > notification message this receiver will delegate the request to 
>>>> the event
>>>> > publisher where you find the set of subscribers with matching filter
>>>> > conditions and execute the mediation sequence and then send the 
>>>> event to
>>>> the
>>>> > specified endpoint.
>>>> >
>>>> > Paul, what do you think about this implementation. I am halfway 
>>>> through
>>>> the
>>>> > implementation and can have a look at this in the weekend :-) I have
>>>> > attached an architecture diagram which explains this concept a 
>>>> little
>>>> more
>>>> > and that explains that the event source itself is now exposed as a
>>>> service
>>>> > to which you can send subscriptions and notifications.
>>>> >
>>>> > Thanks,
>>>> > Ruwan
>>>> >
>>>> > On Fri, Sep 12, 2008 at 9:15 AM, Paul Fremantle <pzfreo@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> Ruwan, AsankaA and I have been building a POC using WS-Eventing

>>>> this
>>>> >> week and we think we have come up with a reasonable model. We've
>>>> >> already iterated several times, and in writing it out I have 
>>>> iterated
>>>> >> beyond what the three of us discussed, so I am expecting more
>>>> >> iterations now.
>>>> >>
>>>> >> What we implemented is a mediator that distributes events based

>>>> on a
>>>> >> filter. The initial code was almost dead simple:
>>>> >>
>>>> >> for (Subscription subs : manager.getSubscribers()) {
>>>> >>                        boolean response =
>>>> >> subs.getFilterMediator().test(mc);
>>>> >>                        if (response) {
>>>> >>                                Endpoint ep = subs.getEndpoint();
>>>> >>                                
>>>> ep.send(getClonedMessageContext(mc));
>>>> >>                        }
>>>> >>                }
>>>> >>
>>>> >> As we implemented the POC it became clear that it was more 
>>>> elegant to
>>>> >> be able to associate a sequence to a particular subscription, and
>>>> >> execute that sequence before sending. This goes a bit beyond the
>>>> >> standard WS-Eventing model, but doesn't seem to contradict it or

>>>> be a
>>>> >> bad fit.
>>>> >>
>>>> >> We also implemented a WS-Eventing subscribe model. Now that is
>>>> >> logically separate, because there might be other ways to subscribe.
>>>> >> For example, you might subscribe by adding an entry in a 
>>>> registry or
>>>> >> using WS-Notification or your own interface. We also have allowed
>>>> >> simple static subscriptions in the synapse.xml model too.
>>>> >>
>>>> >> So the mediator itself is really simple - it only needs to get 
>>>> access
>>>> >> to some kind of thing that manages the subscriptions that can 
>>>> give it
>>>> >> a list of subscribers. In WS-Eventing an "Event Source" is 
>>>> something
>>>> >> that emits events. Effectively our mediator is therefore an event
>>>> >> source. So effectively the event source name is how you 
>>>> reference the
>>>> >> manager that gives you the list of subscribers:
>>>> >>
>>>> >> <sequence>
>>>> >>    <event-source-publisher event-source-name="name"/>
>>>> >> </sequence>
>>>> >>
>>>> >> Now how do you define these event sources. Well we want a new top
>>>> >> level child of <definitions> that is configured at start time.
And
>>>> >> this defines an event-source, and also configures how the
>>>> >> subscriptions can happen.
>>>> >>
>>>> >> <definitions>
>>>> >>  <eventSource name="blah">
>>>> >>     <subscriptionManager
>>>> >> 
>>>> class="org.apache.synapse.events.DefaultInMemorySubscriptionManager">
>>>> >>        <property name="blah">some xml prop</property>
>>>> >>        <property name="other" value="some text property"/>
>>>> >>     </subscriptionManager>
>>>> >>     <subscription id="static1">
>>>> >>        <filter....>
>>>> >>        <sequence...>
>>>> >>        <endpoint..>
>>>> >>     </subscription>
>>>> >>     <subscription...>
>>>> >>     <wsEventing>
>>>> >>        <eventSourceService name="myEventSource">
>>>> >>            <same subchildren of proxy go here>
>>>> >>        </eventSourceService>
>>>> >>        <subscriptionManagerService name="myEventSubManager">
>>>> >>             <same subchildren of proxy go here>
>>>> >>        </subscriptionManager>
>>>> >>     </wsEventing>
>>>> >> <eventSource>
>>>> >>
>>>> >> Lets go through this:
>>>> >> Each event source has a subscription manager. This is a class that
>>>> >> keeps track of subscriptions. Here are some examples: a 
>>>> transient in
>>>> >> memory one. A database backed persistent one. A registry backed
>>>> >> read-only one. A registry backed read-write one. The class must
>>>> >> implement a simple interface:
>>>> >> public interface SubscriptionManager {
>>>> >>        List<Subscription> getSubscribers();
>>>> >>        Subscription getSubscription(String id);
>>>> >>        String addSubscription(Subscription subs);
>>>> >>        boolean deleteSubscription(String id);
>>>> >>
>>>> >> }
>>>> >> The subscriptionManager instance is injected with config 
>>>> properties at
>>>> >> startup just like other things are (tasks, class mediators, etc).
>>>> >> These might contain the JDBC connection parameters or the URL of

>>>> the
>>>> >> registry.
>>>> >>
>>>> >> Next come static subscriptions. These are added into the 
>>>> subscription
>>>> >> manager by synapse. That happens once at startup.
>>>> >>
>>>> >> The next piece is WSEventing specific, but there could be other
>>>> >> children for notification etc. Here I'm not 100% sure that we 
>>>> need to
>>>> >> separate the EventSourceService from the 
>>>> SubscriptionManagerService.
>>>> >> In WS-Eventing it says these can be the same endpoint or different.
>>>> >> Basically the configuration of these is the same as a proxy, 
>>>> allowing
>>>> >> configuration of security etc for this endpoint.
>>>> >>
>>>> >> We certainly haven't done everything. We haven't handled expiry,
>>>> >> though . We haven't thought about other deliveryModes. We haven't
>>>> >> dealt with efficiently handling evaluating multiple subscriptions
>>>> >> against a single message at once. We have simply re-used the 
>>>> existing
>>>> >> filtermediator code to implement XPath and Xpath/Regex filters 
>>>> as is
>>>> >> (we can be much more efficient, for Xpaths, by e.g. using DanD's

>>>> SXC
>>>> >> code which can evaluate multiple Xpaths on a single message). 
>>>> But its
>>>> >> not a bad start.
>>>> >>
>>>> >> I'd really appreciate if we have the right overall structure. I

>>>> did a
>>>> >> first cut of code, but Ruwan is tidying it up right now, so 
>>>> expect a
>>>> >> check-in soon.
>>>> >>
>>>> >> Thanks
>>>> >> Paul
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Paul Fremantle
>>>> >> Co-Founder and CTO, WSO2
>>>> >> Apache Synapse PMC Chair
>>>> >> OASIS WS-RX TC Co-chair
>>>> >>
>>>> >> blog: http://pzf.fremantle.org
>>>> >> paul@wso2.com
>>>> >>
>>>> >> "Oxygenating the Web Service Platform", www.wso2.com
>>>> >>
>>>> >> 
>>>> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>>>> >> For additional commands, e-mail: dev-help@synapse.apache.org
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Ruwan Linton
>>>> > http://wso2.org - "Oxygenating the Web Services Platform"
>>>> > http://ruwansblog.blogspot.com/
>>>> >
>>>> > 
>>>> ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>>>> > For additional commands, e-mail: dev-help@synapse.apache.org
>>>> >
>>>>
>>>>
>>>>
>>>> -- 
>>>> Paul Fremantle
>>>> Co-Founder and CTO, WSO2
>>>> Apache Synapse PMC Chair
>>>> OASIS WS-RX TC Co-chair
>>>>
>>>> blog: http://pzf.fremantle.org
>>>> paul@wso2.com
>>>>
>>>> "Oxygenating the Web Service Platform", www.wso2.com
>>>>
>>>>
>>>>
>>>> -- 
>>>> Paul Fremantle
>>>> Co-Founder and CTO, WSO2
>>>> Apache Synapse PMC Chair
>>>> OASIS WS-RX TC Co-chair
>>>>
>>>> blog: http://pzf.fremantle.org
>>>> paul@wso2.com
>>>>
>>>> "Oxygenating the Web Service Platform", www.wso2.com
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>>>> For additional commands, e-mail: dev-help@synapse.apache.org
>>>>
>>>>
>>>
>>>
>>> -- 
>>> Ruwan Linton
>>> http://wso2.org - "Oxygenating the Web Services Platform"
>>> http://ruwansblog.blogspot.com/
>>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
>> For additional commands, e-mail: dev-help@synapse.apache.org
>>
>>
>
> -- 
> Asankha C. Perera
>
> WSO2 - http://wso2.org
> http://esbmagic.blogspot.com
>

-- 
Asankha C. Perera

WSO2 - http://wso2.org
http://esbmagic.blogspot.com


Mime
View raw message