activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiram Chirino" <hi...@hiramchirino.com>
Subject Re: KahaDB Protocol Buffers and the future of ActiveMQ
Date Thu, 02 Oct 2008 17:14:58 GMT
On Thu, Oct 2, 2008 at 12:38 PM, James Strachan
<james.strachan@gmail.com> wrote:
> 2008/10/2 Hiram Chirino <hiram@hiramchirino.com>:
>> Hi Guys,
>>
>> I've lately been working a new persistence store in the sandbox.  You
>> guys can check it out at:
>> https://svn.apache.org/repos/asf/activemq/sandbox/kahadb
>>
>> It's similar to the default AMQ store that is used by default today in
>> ActiveMQ 5.x, except that it fixes several short commings that we have
>> noticed in in the AMQ store.  This new store uses a transaction log,
>> but indexes the messages using BTrees which stay consistent on
>> restarts which means that store recovery times are very short even
>> when there are many messages stored in the database. This work is
>> approaching a stable point and I think that this should become the
>> default message store for ActiveMQ 6.0.  We need to start beating on
>> this to make sure it's rock solid.
>>
>> While doing this bit of work, I decided to experiment with using
>> Google protocol buffers to encode the transaction log records and it
>> seems to have worked out well.  I think that we should
>> research/evaluate using protocol buffer based default wire format for
>> ActiveMQ.
>
> Agreed! I had a little experiment using PB for the WireFormat and was
> very impressed by the speed.
>
>>   In addition to being able to code generate marshallers for
>> many languages, I think we may get some substantial performance
>> improvements from using protocol buffers.  So I'm going to create a
>> new branch in the sandbox to experiment with changing out the
>> wireformat.  Hopefully, the performance gains do manifest themselves
>> and we can work on merging those changes back to trunk.
>
> Great!
>
> One other thing I love about PB is that (assuming PB doesn't ever
> change its wire format and we are careful never to make a backwards
> incompatible change to existing PB commands we marshal) any PB client
> should be able to read all the messages - even if it cant understand
> some of the new fields. This is a major help! While it might not mean
> that all clients always work perfectly with all broker versions; it'll
> definitely help the situation.
>
>
>> But then this would impact ActiveMQ .NET and CPP....  So I'd be good
>> if the .NET and CPP folks could comment on what they think of the
>> google protocol buffer stuff that is available for their languages.
>
> Agreed.
>
> We should probably keep the old OpenWire and new PB in parallel at
> least until 7.0 where we can deprecate the old OpenWire marshalling if
> we find PB is really good
>

Yeah, a major part of the experiment will be how to integrate PB while
preserving the old OpenWire marshaling support.  Not sure yet if it
will possible to do both efficiently.


> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>



-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Open Source SOA
http://open.iona.com

Mime
View raw message