Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Received-SPF: pass (nike.apache.org: domain of chirino@gmail.com designates
 74.125.46.158 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=mFfdJOXv1w0Di05Ac5+9VLRLQxur9XAQBfzL2fUuwh/KRX3CjdoHoTyaqNv2OW/PWn
         0Vf3x+uoi2Ouex5SnP5viMiU5GUOd+O0jojkt0ZQlOaRlp5U+4xUmVhHnpILUn0YpJCI
         0YZsWIpqfAYSFkqaP+BnmCwkGRrxazfDeyY8Q=
MIME-Version: 1.0
In-Reply-To: <21980708.post@talk.nabble.com>
References: <af2843cd0902111234i77febc4dh723c24f471c0d099@mail.gmail.com>
	 <21980708.post@talk.nabble.com>
Date: Thu, 12 Feb 2009 14:14:03 -0500
Message-ID: <af2843cd0902121114y4f37ae43xe0c64958d9b9719f@mail.gmail.com>
Subject: Re: New sandbox activemq-flow module
From: Hiram Chirino <chirino@gmail.com>
To: dev@activemq.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

FYI.. I just udpated the module to use the ActiveMQ transports so that
way it's testing against real IO.
The wire protocol is just Object Serialization based so it's not very
optimized but at least it should help
us see how the thread models interact.

On Thu, Feb 12, 2009 at 12:24 PM, Colin MacNaughton
<colin.macnaughton@gmail.com> wrote:
>
> Hi Everyone,
>
> As this is my first post to the dev list, I thought I might first share a
> little bit about my background. I started in messaging nine years ago at
> Progress Software working with SonicMQ. With the recent merger between IONA
> and Progress Software, I've happily been given the opportunity to begin
> working with ActiveMQ. I'm very much looking forward to sharing my messaging
> experience from SonicMQ and learning more from the impressive community
> around ActiveMQ.
>
> Anyway as Hiram mentioned I've been working with him on the activemq-flow
> module. One of the things that I find most interesting about this experiment
> is that it tries to boil messaging down into the management and control of
> message flows between sources and sinks. When thought of this way a message
> broker or network of brokers can be represented in a directed flow graph
> with queues as the vertices and message flows as the directed edges. Of
> course, we all know there is a lot more to messaging than this, but to me
> the concept of a flow graph presents some convenient advantages in terms of
> visibility (metrics and connectivity relationships, bottleneck detection) as
> well as management (policies, prioritization etc) to the users and
> administrators of a messaging system.
>
> I'm currently working on the dispatching component of this that Hiram
> mentioned in which we are trying to reduce threading contention between
> queues (sources and sinks). The theory is that by using the flow graph we
> can partition unrelated subcomponents of the graph onto different threads,
> thereby reducing thread contention between unrelated resources, and load
> balance the work amongst a small number of threads using metrics culled from
> the graph. The graphing/metrics component is not really there at the moment
> because the graphing library we were using wasn't really Apache friendly, so
> if anyone knows of a good graphing library please let me know!
>
> Looking forward to working with all of you, Colin
>
> colin.macnaughton@gmail.com
>
>
>
> Hiram Chirino wrote:
>>
>> Hi,
>>
>> Just wanted to let you know about a new activemq-flow little module
>> that I added to the ActiveMQ sandbox.  It's a little experiment in
>> trying to do more efficient flow control and thread handling in todays
>> multi-core world.  Its checked in at:
>> https://svn.apache.org/repos/asf/activemq/sandbox/activemq-flow
>>
>> The 2 major bits of it are basically:
>>   1) an event based thread/dispatching model which tries to avoid
>> cross thread contention
>>   2) A flow control model where dispatching occurs across flow
>> controlled sinks and sources.
>>
>> The flow control bits allow us to more easily implement things that
>> are missing in the current ActiveMQ like priority messaging and to do
>> better broker memory sizing based on connections to the broker.
>>
>> Up to this point this has mostly been a collaborative experiment
>> between Colin Macnaughton (my highly esteemed coworker)  and myself,
>> but I think it's a good time open this out to the rest of the
>> community to see if anyone would be interested in helping out in this
>> little experiment.
>>
>> Things that still need to get done are:
>> - clean up / consolidate some of the unused bits.  For example
>> SingleFlowPriorityQueue should get replaced with the
>> ExclusivePriorityQueue
>> - get the clustering working with the priority queues
>> - Update the mock test cases so it does real IO
>>
>> I think once we get there, then we can start to do better apple to
>> apples comparisons to the existing broker architecture in terms of
>> performance, memory usage, and cpu usage.
>>
>> --
>> Regards,
>> Hiram
>>
>> Blog: http://hiramchirino.com
>>
>> Open Source SOA
>> http://fusesource.com/
>>
>>
>
> --
> View this message in context: http://www.nabble.com/New-sandbox-activemq-flow-module-tp21963659p21980708.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Open Source SOA
http://fusesource.com/