couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From J Chris Anderson <jch...@apache.org>
Subject Re: data recovery tool progress
Date Mon, 09 Aug 2010 18:59:15 GMT
I have described a generalized repair algorithm which should work even for CouchDB's that have
been restarted multiple times leaving orphaned data in the middle of the file (and valid headers
after it)

Adam Kocoloski has offered to write the "for Node in BtreeNodes" part and Volker Mische and
I are working on collecting the optimal set of BtreeNodes.

http://wiki.couchone.com/page/recovery-tool (see the last section "Generalized algorithm")

If you want to write any of this, also write it along side us and we will compare notes and
help each other out on IRC.

Chris


On Aug 9, 2010, at 11:53 AM, Filipe David Manana wrote:

> For the record (and people not on IRC), the code at:
> 
> http://github.com/fdmanana/couchdb/commits/db_repair
> 
> is working for at least simple cases. Use
> couch_db_repair:repair(DbNameAsString).
> There's one TODO:  update the reduce values for the by_seq and by_id BTrees.
> 
> If anyone wants to give some help on this, your welcome.
> 
> On Mon, Aug 9, 2010 at 6:12 PM, Mikeal Rogers <mikeal.rogers@gmail.com>wrote:
> 
>> I'm starting to create a bunch of test db files that expose this bug under
>> different conditions like multiple restarts, across compaction, variances
>> in
>> updates the might cause conflict, etc.
>> 
>> http://github.com/mikeal/couchtest
>> 
>> The README outlines what was done to the db's and what needs to be
>> recovered.
>> 
>> -Mikeal
>> 
>> On Mon, Aug 9, 2010 at 9:33 AM, Filipe David Manana <fdmanana@apache.org
>>> wrote:
>> 
>>> On Mon, Aug 9, 2010 at 5:22 PM, Robert Newson <robert.newson@gmail.com
>>>> wrote:
>>> 
>>>> Doesn't this bit;
>>>> 
>>>> -        Db#db{waiting_delayed_commit=nil};
>>>> +        Db;
>>>> +        % Db#db{waiting_delayed_commit=nil};
>>>> 
>>>> revert the bug fix?
>>>> 
>>> 
>>> That's intentional, for my local testing.
>>> That patch isn't obviously anything close to final, it's too experimental
>>> yet.
>>> 
>>>> 
>>>> B.
>>>> 
>>>> On Mon, Aug 9, 2010 at 5:09 PM, Jan Lehnardt <jan@apache.org> wrote:
>>>>> Hi All,
>>>>> 
>>>>> Filipe jumped in to start working on the recovery tool, but he isn't
>>> done
>>>> yet.
>>>>> 
>>>>> Here's the current patch:
>>>>> 
>>>>> http://www.friendpaste.com/4uMngrym4r7Zz4R0ThSHbz
>>>>> 
>>>>> it is not done and very early, but any help on this is greatly
>>>> appreciated.
>>>>> 
>>>>> The current state is (in Filipe's words):
>>>>> - i can detect that a file needs repair
>>>>> - and get the last btree roots from it
>>>>> - "only" missing: get last db seq num
>>>>> - write new header
>>>>> - and deal with the local docs btree (if exists)
>>>>> 
>>>>> Thanks!
>>>>> Jan
>>>>> --
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Filipe David Manana,
>>> fdmanana@apache.org
>>> 
>>> "Reasonable men adapt themselves to the world.
>>> Unreasonable men adapt the world to themselves.
>>> That's why all progress depends on unreasonable men."
>>> 
>> 
> 
> 
> 
> -- 
> Filipe David Manana,
> fdmanana@apache.org
> 
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."


Mime
View raw message