activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: Stomp/Perl usage?
Date Fri, 14 Jul 2006 08:37:27 GMT
On 7/13/06, Peter Leonard <pete@peteleonard.com> wrote:
> James Strachan wrote:
> > On 7/13/06, Peter Leonard <pete@peteleonard.com> wrote:
> >
> >>
> >> Hi Everyone,
> >>
> >> I've been researching ActiveMQ as a potential solution for a system I'm
> >> working on, and I was hoping for a little feedback from the AMQ
> >> community.
> >>
> >> Is anyone out there using AMQ to support applications written in Perl?
> >> I've found the Stomp/Perl client, but it's described as being of
> >> Alpha-quality, and it doesn't seem like there's much in terms of
> >> community or support specifically for it.
> >>
> >> Assuming that the Perl API for Stomp is in need of some TLC (not an
> >> insurmountable issue on my end), how is the general Stomp interface at
> >> this point?
> >
> >
> > Stomp support is good; we've lots of clients so far and the protocol
> > has been well and trully hammered...
> >
> > http://activemq.org/site/cross-language-clients.html
> >
> > Though there are sometimes some gremlins with certain clients; but the
> > clients are generally simple enough to be easy to fix if you hit any
> > issues. (e.g. the perl client is quite small and pretty simple code to
> > fix if there's anything you need changing etc).
> >
>
> Sounds good - is there anyone out there using the Perl client that could
> shed any light on the situation WRT Perl?  I've perused the client
> source, and it's extremely spartan - I'm sure it works, but it could
> definitely use some polishing.  :)

:)

Given the relative simplicity of the clients in perl, python, ruby and
other scripting languages (and stomp itself being quite simple too)
its probably best to just fire questions here about the stomp commands
that are exchanged and we can all dive in and help. Particularly as
folks like me are pretty clueless when it comes to Perl :)

Incidentally one of the most advanced scripting clients is probably
the ruby one; if you can read Ruby code its well worth taking a look
at etc...

http://stomp.codehaus.org/Ruby


> To confirm - all the Stomp clients are implementing what's documented here:
>
> http://stomp.codehaus.org/Protocol
>
> Is there anything beyond that doc?  For example, failover (see further
> commentary below).

So the above is the only real doc - though there's some more detail on
how to use all the extra features of ActiveMQ and Stomp here

http://incubator.apache.org/activemq/stomp.html

Failover is just a matter of detecting a socket failed and
reconnecting again to one of maybe a list of sockets. On a
reconnection you really should re-subscribe to all the things you were
subscribed to before and resend any in-flight sends/acks.


> I can see the desire to implement both failover & perhaps auto-discovery
> in the Perl client (in addition, setting it up as an easily-installable
> Perl module, like any other CPAN module).  Are there other features in
> the other Stomp clients that would be desireable in the Perl client?

I think those are the big ones. Discovery is the trickiest one I think.


One other thing we added to the ruby client was being able to replay
transactions on the client to preserve order. (If you look at the code
its probably quite self explanatory as its a really small amount of
code).

Basically if you are using a transaction on the client, keep the
messages around until the commit. If the transaction is rolled back,
just redeliver the same messages again. This avoids closing the
consumer and the messages all being redispatched again and causing
order to be broken etc (plus its much faster too).

BTW we should maybe move this over to the dev list :)


> >> (Some quick background - the system requirements would be approximately
> >> 500 messages/second over an HA environment, supporting message
> >> persistance for potentially several hours, if possible.  Early reading
> >> with AMQ suggests that setting up a cluster would be the way to go).
> >
> >
> > Yeah - the perl would connect to a Master which could failover to a
> > Slave...
> >
> > http://activemq.org/site/masterslave.html
> >
> > the only thing that needs adding is the ability to failver in the Perl
> > client which I don't think is there yet.
> >
>
> Funny you should bring that up.  :)
>
> I'm a little confused with the AMQ docs with respect to MasterSlave
> versus clustering versus other topologies.
>
> Some of the docs talk about MasterSlave as the way to implement HA
> setups, but then there are other docs talking about clustering brokers
> to create a messaging fabric, which suggests a network of equals.
>
> To add to the confusion, the website differentiates between "Broker
> Clusters" and "Networks of Brokers", talking about them as two different
> things, but yet the paragraphs describing the two terms seem somewhat alike:
>
> http://incubator.apache.org/activemq/clustering.html
>
> Some insight would be appreciated - perhaps the answer to this question
> would be best -

Sorry about that; its hard keeping all the docs up to date and
sometimes things use different terminology to confuse folks. I hope we
can tidy up the clustering docs to make things absolutely clear
(clustering can be quite confusing).

Basically Master/Slave does the replicaton thing; where messages are
replicated to the slave so if you loose the entire hardware/data
centre running the master you can carry on without loosing any
messages.

You can also create a 'fabric' of brokers using store and forward
(i.e. networks of brokers). This allows clients to connect to any
broker and failover to another broker (just as clients do with
master/slave when the master dies).


> If you had to set up AMQ to support 500 messages/second, in HA fashion,
> with message persistance  (so clients could retrieve old messages) and
> you had a network of 10-15 machines that would run services interacting
> with AMQ, how would you set it up?

A pair of brokers in a Master/Slave pair and everyone else a client of
the broker(s). 500/second is fairly low volume. On modern blades
(particularly with things like opterons) its only in the 10,000/second
range that I'd start thinking about networks of brokers. YMMV

http://incubator.apache.org/activemq/performance.html


> I know that some of this is more of a philosophical battle, and I'm not
> trying to start a holy war of any kind.  I am personally a little
> hesitant about the Master/Slave relationship, if only because I know it
> from the RDBMS world, and I know the hell that can arise when a Master
> DB blows up.  If a messaging fabric makes that problem go away, I'm a
> happy camper.

The only real reason for master/slave is to replicate messages to be
able to deal with hardware failures. If you are not using persistent
messaging or trust your hardware/disks or have a SAN (or reliable
shared drive of some kind) then you don't need to use it - you can
just have a number of brokers. Networks are only required when you
want to store and forward between brokers. You could even just have
separate brokers - and just get your consumers to connect to every
broker (there's a feature request to support that out of the
box...)...

http://issues.apache.org/activemq/browse/AMQ-816

the only problem with having networks of brokers or stand alone
brokers is if a broker goes down; any messages on the disk of the box
stay there until the broker is restarted. Whereas with Master/Slave
the messages are all replicated so you can loose an entire box/data
centre and still keep going.
-- 

James
-------
http://radio.weblogs.com/0112098/

Mime
View raw message