activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Colin MacNaughton <colin.macnaugh...@gmail.com>
Subject Re: New sandbox activemq-flow module
Date Thu, 12 Feb 2009 17:24:29 GMT

Hi Everyone, 

As this is my first post to the dev list, I thought I might first share a
little bit about my background. I started in messaging nine years ago at
Progress Software working with SonicMQ. With the recent merger between IONA
and Progress Software, I've happily been given the opportunity to begin
working with ActiveMQ. I'm very much looking forward to sharing my messaging
experience from SonicMQ and learning more from the impressive community
around ActiveMQ.  

Anyway as Hiram mentioned I've been working with him on the activemq-flow
module. One of the things that I find most interesting about this experiment
is that it tries to boil messaging down into the management and control of
message flows between sources and sinks. When thought of this way a message
broker or network of brokers can be represented in a directed flow graph
with queues as the vertices and message flows as the directed edges. Of
course, we all know there is a lot more to messaging than this, but to me
the concept of a flow graph presents some convenient advantages in terms of
visibility (metrics and connectivity relationships, bottleneck detection) as
well as management (policies, prioritization etc) to the users and
administrators of a messaging system. 

I'm currently working on the dispatching component of this that Hiram
mentioned in which we are trying to reduce threading contention between
queues (sources and sinks). The theory is that by using the flow graph we
can partition unrelated subcomponents of the graph onto different threads,
thereby reducing thread contention between unrelated resources, and load
balance the work amongst a small number of threads using metrics culled from
the graph. The graphing/metrics component is not really there at the moment
because the graphing library we were using wasn't really Apache friendly, so
if anyone knows of a good graphing library please let me know!

Looking forward to working with all of you, Colin

colin.macnaughton@gmail.com



Hiram Chirino wrote:
> 
> Hi,
> 
> Just wanted to let you know about a new activemq-flow little module
> that I added to the ActiveMQ sandbox.  It's a little experiment in
> trying to do more efficient flow control and thread handling in todays
> multi-core world.  Its checked in at:
> https://svn.apache.org/repos/asf/activemq/sandbox/activemq-flow
> 
> The 2 major bits of it are basically:
>   1) an event based thread/dispatching model which tries to avoid
> cross thread contention
>   2) A flow control model where dispatching occurs across flow
> controlled sinks and sources.
> 
> The flow control bits allow us to more easily implement things that
> are missing in the current ActiveMQ like priority messaging and to do
> better broker memory sizing based on connections to the broker.
> 
> Up to this point this has mostly been a collaborative experiment
> between Colin Macnaughton (my highly esteemed coworker)  and myself,
> but I think it's a good time open this out to the rest of the
> community to see if anyone would be interested in helping out in this
> little experiment.
> 
> Things that still need to get done are:
> - clean up / consolidate some of the unused bits.  For example
> SingleFlowPriorityQueue should get replaced with the
> ExclusivePriorityQueue
> - get the clustering working with the priority queues
> - Update the mock test cases so it does real IO
> 
> I think once we get there, then we can start to do better apple to
> apples comparisons to the existing broker architecture in terms of
> performance, memory usage, and cpu usage.
> 
> -- 
> Regards,
> Hiram
> 
> Blog: http://hiramchirino.com
> 
> Open Source SOA
> http://fusesource.com/
> 
> 

-- 
View this message in context: http://www.nabble.com/New-sandbox-activemq-flow-module-tp21963659p21980708.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.


Mime
View raw message