activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiram Chirino" <>
Subject Re: KahaDB Protocol Buffers and the future of ActiveMQ
Date Thu, 02 Oct 2008 17:14:58 GMT
On Thu, Oct 2, 2008 at 12:38 PM, James Strachan
<> wrote:
> 2008/10/2 Hiram Chirino <>:
>> Hi Guys,
>> I've lately been working a new persistence store in the sandbox.  You
>> guys can check it out at:
>> It's similar to the default AMQ store that is used by default today in
>> ActiveMQ 5.x, except that it fixes several short commings that we have
>> noticed in in the AMQ store.  This new store uses a transaction log,
>> but indexes the messages using BTrees which stay consistent on
>> restarts which means that store recovery times are very short even
>> when there are many messages stored in the database. This work is
>> approaching a stable point and I think that this should become the
>> default message store for ActiveMQ 6.0.  We need to start beating on
>> this to make sure it's rock solid.
>> While doing this bit of work, I decided to experiment with using
>> Google protocol buffers to encode the transaction log records and it
>> seems to have worked out well.  I think that we should
>> research/evaluate using protocol buffer based default wire format for
>> ActiveMQ.
> Agreed! I had a little experiment using PB for the WireFormat and was
> very impressed by the speed.
>>   In addition to being able to code generate marshallers for
>> many languages, I think we may get some substantial performance
>> improvements from using protocol buffers.  So I'm going to create a
>> new branch in the sandbox to experiment with changing out the
>> wireformat.  Hopefully, the performance gains do manifest themselves
>> and we can work on merging those changes back to trunk.
> Great!
> One other thing I love about PB is that (assuming PB doesn't ever
> change its wire format and we are careful never to make a backwards
> incompatible change to existing PB commands we marshal) any PB client
> should be able to read all the messages - even if it cant understand
> some of the new fields. This is a major help! While it might not mean
> that all clients always work perfectly with all broker versions; it'll
> definitely help the situation.
>> But then this would impact ActiveMQ .NET and CPP....  So I'd be good
>> if the .NET and CPP folks could comment on what they think of the
>> google protocol buffer stuff that is available for their languages.
> Agreed.
> We should probably keep the old OpenWire and new PB in parallel at
> least until 7.0 where we can deprecate the old OpenWire marshalling if
> we find PB is really good

Yeah, a major part of the experiment will be how to integrate PB while
preserving the old OpenWire marshaling support.  Not sure yet if it
will possible to do both efficiently.

> --
> James
> -------
> Open Source Integration



Open Source SOA

View raw message