synapse-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaliya Ekanayake" <jnekanay...@gmail.com>
Subject Re: Proposal to implement ws-eventing and an event distribution model in Synapse
Date Sat, 27 Sep 2008 04:25:32 GMT
Hi Ruwan,

I have a little comments about the architecture that you have shown. May be you have already
think about this.
According to the diagram the main thread driving the entire message path is the incoming thread.


>From my experiance to get a system of this nature to scale well, we need to break this
continouation of threads.
For example, the incoming messages can be put into a Queue and the "broker" can keep processing
the messages in the Queue.
This way we can improve the concurrency in processing.

Thanks,
Jaliya


  ----- Original Message ----- 
  From: Ruwan Linton 
  To: dev@synapse.apache.org 
  Sent: Friday, September 12, 2008 5:01 AM
  Subject: Re: Proposal to implement ws-eventing and an event distribution model in Synapse


  Paul,

  Very nice explanation of the concepts that we have been trying to put together into the
code. Let me add some more to your explanation and refine the configuration a bit more.

  <eventSource name="blah">
      <subscriptionManager class="org.apache.synapse.events.DefaultInMemorySubscriptionManager">
          <property name="blah">some xml prop</property>
          <property name="other" value="some text property"/>
      </subscriptionManager>     
      <staticSubscriptions>
          <subscription id="static1">
              <filter..../>
              <sequence.../>
              <endpoint../>
          </subscription>*
      <staticSubscriptions>?
  <eventSource/>

  Here I am getting rid of the wsEventing configuration element where you specify the subscription
service and the event source service. So my idea is we can extend the proxy services model
here and create a new EventingMessageReceiver, which listens for all the requests coming to
this event source. (I must also say at this point event source is now a service inside synapse
and that fits with the model of extending the proxy service behavior)

  This EventingMessageReceiver knows how to filter out the the subscription messages from
the notification messages and it uses the specified subscription manager if it is a subscription
request, and if it is a notification message this receiver will delegate the request to the
event publisher where you find the set of subscribers with matching filter conditions and
execute the mediation sequence and then send the event to the specified endpoint.

  Paul, what do you think about this implementation. I am halfway through the implementation
and can have a look at this in the weekend :-) I have attached an architecture diagram which
explains this concept a little more and that explains that the event source itself is now
exposed as a service to which you can send subscriptions and notifications.

  Thanks,
  Ruwan


  On Fri, Sep 12, 2008 at 9:15 AM, Paul Fremantle <pzfreo@gmail.com> wrote:

    Ruwan, AsankaA and I have been building a POC using WS-Eventing this
    week and we think we have come up with a reasonable model. We've
    already iterated several times, and in writing it out I have iterated
    beyond what the three of us discussed, so I am expecting more
    iterations now.

    What we implemented is a mediator that distributes events based on a
    filter. The initial code was almost dead simple:

    for (Subscription subs : manager.getSubscribers()) {
                           boolean response = subs.getFilterMediator().test(mc);
                           if (response) {
                                   Endpoint ep = subs.getEndpoint();
                                   ep.send(getClonedMessageContext(mc));
                           }
                   }

    As we implemented the POC it became clear that it was more elegant to
    be able to associate a sequence to a particular subscription, and
    execute that sequence before sending. This goes a bit beyond the
    standard WS-Eventing model, but doesn't seem to contradict it or be a
    bad fit.

    We also implemented a WS-Eventing subscribe model. Now that is
    logically separate, because there might be other ways to subscribe.
    For example, you might subscribe by adding an entry in a registry or
    using WS-Notification or your own interface. We also have allowed
    simple static subscriptions in the synapse.xml model too.

    So the mediator itself is really simple - it only needs to get access
    to some kind of thing that manages the subscriptions that can give it
    a list of subscribers. In WS-Eventing an "Event Source" is something
    that emits events. Effectively our mediator is therefore an event
    source. So effectively the event source name is how you reference the
    manager that gives you the list of subscribers:

    <sequence>
       <event-source-publisher event-source-name="name"/>
    </sequence>

    Now how do you define these event sources. Well we want a new top
    level child of <definitions> that is configured at start time. And
    this defines an event-source, and also configures how the
    subscriptions can happen.

    <definitions>
     <eventSource name="blah">
        <subscriptionManager
    class="org.apache.synapse.events.DefaultInMemorySubscriptionManager">
           <property name="blah">some xml prop</property>
           <property name="other" value="some text property"/>
        </subscriptionManager>
        <subscription id="static1">
           <filter....>
           <sequence...>
           <endpoint..>
        </subscription>
        <subscription...>
        <wsEventing>
           <eventSourceService name="myEventSource">
               <same subchildren of proxy go here>
           </eventSourceService>
           <subscriptionManagerService name="myEventSubManager">
                <same subchildren of proxy go here>
           </subscriptionManager>
        </wsEventing>
    <eventSource>

    Lets go through this:
    Each event source has a subscription manager. This is a class that
    keeps track of subscriptions. Here are some examples: a transient in
    memory one. A database backed persistent one. A registry backed
    read-only one. A registry backed read-write one. The class must
    implement a simple interface:
    public interface SubscriptionManager {
           List<Subscription> getSubscribers();
           Subscription getSubscription(String id);
           String addSubscription(Subscription subs);
           boolean deleteSubscription(String id);

    }
    The subscriptionManager instance is injected with config properties at
    startup just like other things are (tasks, class mediators, etc).
    These might contain the JDBC connection parameters or the URL of the
    registry.

    Next come static subscriptions. These are added into the subscription
    manager by synapse. That happens once at startup.

    The next piece is WSEventing specific, but there could be other
    children for notification etc. Here I'm not 100% sure that we need to
    separate the EventSourceService from the SubscriptionManagerService.
    In WS-Eventing it says these can be the same endpoint or different.
    Basically the configuration of these is the same as a proxy, allowing
    configuration of security etc for this endpoint.

    We certainly haven't done everything. We haven't handled expiry,
    though . We haven't thought about other deliveryModes. We haven't
    dealt with efficiently handling evaluating multiple subscriptions
    against a single message at once. We have simply re-used the existing
    filtermediator code to implement XPath and Xpath/Regex filters as is
    (we can be much more efficient, for Xpaths, by e.g. using DanD's SXC
    code which can evaluate multiple Xpaths on a single message). But its
    not a bad start.

    I'd really appreciate if we have the right overall structure. I did a
    first cut of code, but Ruwan is tidying it up right now, so expect a
    check-in soon.

    Thanks
    Paul


    --
    Paul Fremantle
    Co-Founder and CTO, WSO2
    Apache Synapse PMC Chair
    OASIS WS-RX TC Co-chair

    blog: http://pzf.fremantle.org
    paul@wso2.com

    "Oxygenating the Web Service Platform", www.wso2.com

    ---------------------------------------------------------------------
    To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
    For additional commands, e-mail: dev-help@synapse.apache.org





  -- 
  Ruwan Linton
  http://wso2.org - "Oxygenating the Web Services Platform"
  http://ruwansblog.blogspot.com/



------------------------------------------------------------------------------


  ---------------------------------------------------------------------
  To unsubscribe, e-mail: dev-unsubscribe@synapse.apache.org
  For additional commands, e-mail: dev-help@synapse.apache.org
Mime
View raw message