directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lucas Theisen <lucasthei...@pastdev.com>
Subject Re: JDBM corruptions...
Date Wed, 14 Oct 2015 18:18:27 GMT
On Wed, Oct 14, 2015 at 9:31 AM, Kiran Ayyagari <kayyagari@apache.org>
wrote:

>
>
> On Wed, Oct 14, 2015 at 9:13 PM, Emmanuel L├ęcharny <elecharny@gmail.com>
> wrote:
>
>> Hi guys,
>>
>> so we have clear case where the JDBM backend get corrupted, up to a
>> point a full reimport of the data is required. Lucas also proved that a
>> test will fail, after haing injected 1000 entries and doing 100 rename
>> (https://issues.apache.org/jira/browse/DIRSERVER-1974).
>>
>> This is more than annoying...
>>
>> I have tested DIRSERVER-1974 with Mavibot, and I can't reproduce the
>> problem, which means the server is safe. The resulting database is 14 Mb
>> big with the 1000 entries being added, which is big, but acceptable.
>>
>> At this point, I wonder if it would not be a safe approach to switch to
>> Mavibot right now ?
>>
>> The pros :
>> - we know that we can't have a database corruption, because each update
>> is a new revision
>> - technically, mavibot is faster than JDBM
>> - we aren't maintaining JDBM, while Mavibot is under development
>>
>> The cons :
>> - Mavibot is still under developement : we still have to add teh
>> cross-btree transactions, which means that if we have a brutal crash
>> during an update, then we may have some inconsistancy in the base
>> (inconsitancy != corruption, but still)
>> - We will still need the global write locking strategy to protect the
>> backend from concurrent reads and write when a update is processed
>> - The database is growing fast due to the limited cleanup we currently do
>>
>> However, in the middle term, Mavibot will bring the following bonuses :
>> - cross btree transaction support (actually, we may even support
>> multiple updates within a single transaction, speeding up the update
>> even more), which will allow us to remove the global RW lock we
>> currently have
>> - bulkloading capacity, increasing teh speed of injecting data by at
>> leats 2 orders of magnitude (server being stopped)
>>
>>
>> So, I'll ask you : what about releasing M21 with Mavibot as a default,
>> but with a possibility for users to still pick JDBM on demand ?
>>
>> I am in favor of that, I have tested it with the server few months ago
> after
> adding free page reclaiming functionality and it was _working_
>
> The addition of entries gets slower after a point of time, but other than
> that
> I have not seen any other issues, this is just from one user pov though.
>
> Server needs to modify a bit to detect the type of storage used in an
> existing partition
> and initialize accordingly.
>
> --
> Kiran Ayyagari
> http://keydap.com
>

I agree, I think it is time we migrate to Mavibot unless there is some
major stumbling block and I think you two are probably the only 2 people
who could speak to that.  So if you are comfortable with the transition,
I'm all for it.

Lucas.

Mime
View raw message