activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: KahaDB Protocol Buffers and the future of ActiveMQ
Date Thu, 02 Oct 2008 16:38:37 GMT
2008/10/2 Hiram Chirino <hiram@hiramchirino.com>:
> Hi Guys,
>
> I've lately been working a new persistence store in the sandbox.  You
> guys can check it out at:
> https://svn.apache.org/repos/asf/activemq/sandbox/kahadb
>
> It's similar to the default AMQ store that is used by default today in
> ActiveMQ 5.x, except that it fixes several short commings that we have
> noticed in in the AMQ store.  This new store uses a transaction log,
> but indexes the messages using BTrees which stay consistent on
> restarts which means that store recovery times are very short even
> when there are many messages stored in the database. This work is
> approaching a stable point and I think that this should become the
> default message store for ActiveMQ 6.0.  We need to start beating on
> this to make sure it's rock solid.
>
> While doing this bit of work, I decided to experiment with using
> Google protocol buffers to encode the transaction log records and it
> seems to have worked out well.  I think that we should
> research/evaluate using protocol buffer based default wire format for
> ActiveMQ.

Agreed! I had a little experiment using PB for the WireFormat and was
very impressed by the speed.

>   In addition to being able to code generate marshallers for
> many languages, I think we may get some substantial performance
> improvements from using protocol buffers.  So I'm going to create a
> new branch in the sandbox to experiment with changing out the
> wireformat.  Hopefully, the performance gains do manifest themselves
> and we can work on merging those changes back to trunk.

Great!

One other thing I love about PB is that (assuming PB doesn't ever
change its wire format and we are careful never to make a backwards
incompatible change to existing PB commands we marshal) any PB client
should be able to read all the messages - even if it cant understand
some of the new fields. This is a major help! While it might not mean
that all clients always work perfectly with all broker versions; it'll
definitely help the situation.


> But then this would impact ActiveMQ .NET and CPP....  So I'd be good
> if the .NET and CPP folks could comment on what they think of the
> google protocol buffer stuff that is available for their languages.

Agreed.

We should probably keep the old OpenWire and new PB in parallel at
least until 7.0 where we can deprecate the old OpenWire marshalling if
we find PB is really good

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Mime
View raw message