Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Content-Type: text/plain; charset=windows-1252
Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\))
Subject: Re: Corrupt database file
From: Robert Samuel Newson <rnewson@apache.org>
In-Reply-To: 
 <CAGxYWpaVSmRxswe8m4SQr6upKAzhebzoKdCXSS_8UP8hbOV0CA@mail.gmail.com>
Date: Fri, 21 Mar 2014 22:44:02 +0000
Content-Transfer-Encoding: quoted-printable
Message-Id: <5019F936-6AA3-4632-991D-A67A6897A89C@apache.org>
References: <1395402372.93156.YahooMailNeo@web171406.mail.ir2.yahoo.com>
 <CAGxYWpbWD0B1RC=+dT_sUpL9+tzeKieNfxD_Bs9zWvLVTFh90g@mail.gmail.com>
 <1395408893.89620.YahooMailNeo@web171401.mail.ir2.yahoo.com>
 <ACEF785E-F1F9-4EFB-BAED-641CECD9A440@apache.org>
 <19917F3A-355F-421D-9BE5-F2B075ECB44C@couchbase.com>
 <CAHdjipLFrGyk7_e=QExYA0Mj-OpygT7EVAZzB_HPiN8z5WEeWg@mail.gmail.com>
 <2350E5F5-51B2-465B-9A54-FAAB435E6E42@couchbase.com>
 <CAGxYWpaVSmRxswe8m4SQr6upKAzhebzoKdCXSS_8UP8hbOV0CA@mail.gmail.com>
To: user@couchdb.apache.org

"If the DB was corrupted because the disk became full, shouldn't the DB =
still be fine but just missing the most recent commits?"

Yes, that=92s the virtue of the append-only nature of database files, =
though the code that detects file corruption happens when the md5 =
checksums fail to verify, it=92s hard to imagine it being a false =
positive.

Did the compaction attempt fail? Can it be replicated? If not, I would =
reluctantly truncate a few meg off the file and see if it can be opened =
(do this when couchdb is not running). The actual corrupted file would =
be useful to couchdb developers so that we could investigate the raw =
data at the corruption site.

What was the disk system here? RAID? filesystem? Would your disk =
controllers reorder writers at all?

B.

On 21 Mar 2014, at 20:19, Tim Tisdall <tisdall@gmail.com> wrote:

> If the DB was corrupted because the disk became full, shouldn't the DB
> still be fine but just missing the most recent commits?  Or would the =
a
> person need to truncate a certain number of bytes off the end of the =
DB to
> get it to read properly?
>=20
> As for JSON file size... I always dump the DB into a GZ file and then =
my
> scripts work on it as a GZ'ed file.  In my case the JSON is 20gb and =
the gz
> file is 3.5gb.  Dealing with the file as a gz adds a little more =
complexity
> to the script you use to process it, though.
>=20
>=20
>=20
> On 21 March 2014 15:46, Jens Alfke <jens@couchbase.com> wrote:
>=20
>>=20
>> On Mar 21, 2014, at 8:43 AM, Alexander Shorin <kxepal@gmail.com> =
wrote:
>>=20
>>> we don't have any knowledge about how it was repacked, what
>>> changes it includes, how it different from original CouchDB 1.2.0 =
and
>>> so on and so forth.
>>=20
>> Is that relevant? Nothing's come up in this thread that depends on =
exact
>> details of CouchDB internals.  Can we focus on the issue at hand, =
namely
>> that the OP has a CouchDB that ran out of disk space and corrupted =
its
>> database and he'd like to recover the data?
>>=20
>> (IIRC, if there were any source changes from stock 1.2 they were =
minor,
>> maybe just around branding. Maybe Jan, Dale, or Filipe remember more =
about
>> it?)
>>=20
>>> CouchBase team probably should knows more about their products.
>>=20
>> Couchbase's forums are not going to respond to support requests for a
>> product that's been discontinued for over two years, from someone =
who's
>> (presumably) not a paying customer.
>>=20
>> --Jens