Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (athena.apache.org: domain of elecharny@gmail.com
 designates 74.125.82.44 as permitted sender)
Message-ID: <4E42B3C5.3050606@gmail.com>
Date: Wed, 10 Aug 2011 18:37:25 +0200
From: Emmanuel Lecharny <elecharny@gmail.com>
Reply-To: elecharny@apache.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6;
 rv:5.0) Gecko/20110624 Thunderbird/5.0
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: Re: Replication & activeMQ
References: <4E42A6F1.6060709@gmail.com>
 <CABzFU-e8p6CpJOHko2A29myqvvxyWd+z077AcKXLgiGqVQi+Vg@mail.gmail.com>
In-Reply-To: 
 <CABzFU-e8p6CpJOHko2A29myqvvxyWd+z077AcKXLgiGqVQi+Vg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 8/10/11 6:16 PM, Kiran Ayyagari wrote:
>
>> Now, we already are saving the mods in a file, in the JournalInterceptor. We
>> just have to implement the recovery system (ie, finding the last entry sent
>> from the file) and send all the following entries. Then we can delete all
>> the entries older than the requested one.
>>
>> I don't think that using our own implementation would be an issue here. I
>> mean, ActiveMQ is a great piece of software, but having to go through tends
>> of options (hundreds?), most of which are totally useless in our case, is a
>> bit overkilling for this version.
>>
>> thoughts ?
>>
> the syncrepl protocol provides a high level of granularity about the
> kind of data that can be
> replicated, so not each mod in this journal is necessarily replicated
> to a client (it can even be
> a serious issue to send to that client based on the sensitivity of data).
>
> This brings the issue of filtering the data that needs to be sent to
> the client, this requires significant
> time for scanning, processing and maintaining the position pointers in
> the monolithic journal.
Don't get me wrong : we will have one journal per replication consumer. 
The filtering will then be done once, on the provider side.
>
> ActiveMQ's core offers all these features so that was a preferred
> choice instead of writing a journal with all the above
> mentioned features, cause implementing such a journal is quite a
> handful of work and our main problem to be solved
> is replication.
>
> Having said that ActiveMQ is definitely an over kill to be used as a
> journal, the main part that we actually need is
> it's journal/store implementation called 'kahadb' but it is easy to
> use through high level message queue interfaces.
>
> I would like to spend some time on 'kahadb' source to see if that can
> be easily embeddable and serves our purpose.
Yes, this is exactly what I was looking at when I saw your mail. That 
may be a very good balance between complexity and ease of use.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com