directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn McKinney <smckin...@apache.org>
Subject Re: JdbmPartition repair
Date Sun, 24 Jan 2016 14:07:58 GMT
Emmanuel, thanks for keeping us informed.  I agree that corruption of data is a show stopper
in terms of a product’s viability.  Can we recreate this issue or is it intermittent?  How
can we help?

Shawn

> On Jan 24, 2016, at 7:39 AM, Emmanuel LĂ©charny <elecharny@gmail.com> wrote:
> 
> Hi guys,
> 
> we have many users complaining about a corrupted JDBM database. As of
> today, we don't have another solution than telling them to reload their
> data, which is all be comfortable. First because it might take ages
> (reloading data is very slow) and also because they might not have a backup.
> 
> Although this is not a frequent scenario, when it happens, it really
> take down any credibility ApacheDS can have.
> 
> Here, we all know that Mavibot will be the solution, but until it's
> available with transaction support, we have to propose a tool that
> restore - of possible - the database.
> 
> Hopefully, Kiran has worked on a tool that does that : the
> partition-plumber. The idea is to intergrate this tool into ApacheDS in
> order to allow users to restore their database in  a simple way. Here is
> what I propose :
> 
> - first, a way to start ApacheDS in a repair mode. That will drip all
> the indexes, and recreate them based on the master table. It might take
> some time, but it will be way better than any solution, and in any case,
> will be faster than a full reload, consider that we will bypass many
> checks. I suggest an option : apacheds -repair. When the server is
> started with that option, the server will restart after having cleaned
> up the database
> - the way to implement it is to add a method in the Partition interface
> : repair(). Not all the partition will need it, so only JdbmPartition
> will actually iumplement it.
> - that method will simply delete (or copy, if we want a backup) all the
> existing indexes (system and users). We then will recreate the indexes
> based on the master table content. There is still a remote risk that the
> master table can be corrupted, but it's unlikely, or at least very rare.
> Actually, the Rdn index is the one which get corrupted most of the time,
> because it get updated many times for each addition, move, rename or
> delete operations.
> 
> I'm currently working on that, and it should be done fast enough (say,
> in less than a week, or even quicker if I have enough time this sundy
> and at night).
> 
> The next step, and I'm also working on that, is to finish Mavibot. The
> problem is that it's a complex piece of code, and it's hard to work on
> it when I just have a couple of hours on evening or during the week-end.
> I'm sorry for that. But we eventually will get it ready !
> 
> 
> Thanks !
> 


Mime
View raw message