directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lecharny <>
Subject Re: Replication & activeMQ
Date Wed, 10 Aug 2011 16:37:25 GMT
On 8/10/11 6:16 PM, Kiran Ayyagari wrote:
>> Now, we already are saving the mods in a file, in the JournalInterceptor. We
>> just have to implement the recovery system (ie, finding the last entry sent
>> from the file) and send all the following entries. Then we can delete all
>> the entries older than the requested one.
>> I don't think that using our own implementation would be an issue here. I
>> mean, ActiveMQ is a great piece of software, but having to go through tends
>> of options (hundreds?), most of which are totally useless in our case, is a
>> bit overkilling for this version.
>> thoughts ?
> the syncrepl protocol provides a high level of granularity about the
> kind of data that can be
> replicated, so not each mod in this journal is necessarily replicated
> to a client (it can even be
> a serious issue to send to that client based on the sensitivity of data).
> This brings the issue of filtering the data that needs to be sent to
> the client, this requires significant
> time for scanning, processing and maintaining the position pointers in
> the monolithic journal.
Don't get me wrong : we will have one journal per replication consumer. 
The filtering will then be done once, on the provider side.
> ActiveMQ's core offers all these features so that was a preferred
> choice instead of writing a journal with all the above
> mentioned features, cause implementing such a journal is quite a
> handful of work and our main problem to be solved
> is replication.
> Having said that ActiveMQ is definitely an over kill to be used as a
> journal, the main part that we actually need is
> it's journal/store implementation called 'kahadb' but it is easy to
> use through high level message queue interfaces.
> I would like to spend some time on 'kahadb' source to see if that can
> be easily embeddable and serves our purpose.
Yes, this is exactly what I was looking at when I saw your mail. That 
may be a very good balance between complexity and ease of use.

Emmanuel L├ęcharny

View raw message