couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikeal Rogers <mikeal.rog...@gmail.com>
Subject Re: data recovery tool progress
Date Mon, 09 Aug 2010 20:39:13 GMT
Ok, I pulled down this code and tested against a database with a ton of
missing writes right before a single restart.

Before restart this was the database:

  {
    db_name: "testwritesdb"
    doc_count: 124969
    doc_del_count: 0
    update_seq: 124969
    purge_seq: 0
    compact_running: false
    disk_size: 54857478
    instance_start_time: "1281384140058211"
    disk_format_version: 5
  }

After restart it was this:

  {
    db_name: "testwritesdb"
    doc_count: 1
    doc_del_count: 0
    update_seq: 1
    purge_seq: 0
    compact_running: false
    disk_size: 54857478
    instance_start_time: "1281384593876026"
    disk_format_version: 5
  }

After repair, it's this:

{
  db_name: "testwritesdb"
  doc_count: 1
  doc_del_count: 0
  update_seq: 124969
  purge_seq: 0
  compact_running: false
  disk_size: 54857820
  instance_start_time: "1281385990193289"
  disk_format_version: 5
  committed_update_seq: 124969
}

All the sequences are there and hitting _all_docs shows all the documents so
why is the doc_count only 1 in the dbinfo?

-Mikeal

On Mon, Aug 9, 2010 at 11:53 AM, Filipe David Manana <fdmanana@apache.org>wrote:

> For the record (and people not on IRC), the code at:
>
> http://github.com/fdmanana/couchdb/commits/db_repair
>
> is working for at least simple cases. Use
> couch_db_repair:repair(DbNameAsString).
> There's one TODO:  update the reduce values for the by_seq and by_id
> BTrees.
>
> If anyone wants to give some help on this, your welcome.
>
> On Mon, Aug 9, 2010 at 6:12 PM, Mikeal Rogers <mikeal.rogers@gmail.com
> >wrote:
>
> > I'm starting to create a bunch of test db files that expose this bug
> under
> > different conditions like multiple restarts, across compaction, variances
> > in
> > updates the might cause conflict, etc.
> >
> > http://github.com/mikeal/couchtest
> >
> > The README outlines what was done to the db's and what needs to be
> > recovered.
> >
> > -Mikeal
> >
> > On Mon, Aug 9, 2010 at 9:33 AM, Filipe David Manana <fdmanana@apache.org
> > >wrote:
> >
> > > On Mon, Aug 9, 2010 at 5:22 PM, Robert Newson <robert.newson@gmail.com
> > > >wrote:
> > >
> > > > Doesn't this bit;
> > > >
> > > > -        Db#db{waiting_delayed_commit=nil};
> > > > +        Db;
> > > > +        % Db#db{waiting_delayed_commit=nil};
> > > >
> > > > revert the bug fix?
> > > >
> > >
> > > That's intentional, for my local testing.
> > > That patch isn't obviously anything close to final, it's too
> experimental
> > > yet.
> > >
> > > >
> > > > B.
> > > >
> > > > On Mon, Aug 9, 2010 at 5:09 PM, Jan Lehnardt <jan@apache.org> wrote:
> > > > > Hi All,
> > > > >
> > > > > Filipe jumped in to start working on the recovery tool, but he
> isn't
> > > done
> > > > yet.
> > > > >
> > > > > Here's the current patch:
> > > > >
> > > > > http://www.friendpaste.com/4uMngrym4r7Zz4R0ThSHbz
> > > > >
> > > > > it is not done and very early, but any help on this is greatly
> > > > appreciated.
> > > > >
> > > > > The current state is (in Filipe's words):
> > > > >  - i can detect that a file needs repair
> > > > >  - and get the last btree roots from it
> > > > >  - "only" missing: get last db seq num
> > > > >  - write new header
> > > > >  - and deal with the local docs btree (if exists)
> > > > >
> > > > > Thanks!
> > > > > Jan
> > > > > --
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Filipe David Manana,
> > > fdmanana@apache.org
> > >
> > > "Reasonable men adapt themselves to the world.
> > >  Unreasonable men adapt the world to themselves.
> > >  That's why all progress depends on unreasonable men."
> > >
> >
>
>
>
> --
> Filipe David Manana,
> fdmanana@apache.org
>
> "Reasonable men adapt themselves to the world.
>  Unreasonable men adapt the world to themselves.
>  That's why all progress depends on unreasonable men."
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message