Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CABvT1DEucB-oZ8-ge1KoBCZu=oboJR9B0vaqBkHv3Nc-W1WT_g@mail.gmail.com>
References: 
 <CANpCgCP1WGMuu=oVHQOgG50zBkgdGdA5FU-Xd8XtN++m9P5fQA@mail.gmail.com>
	<CABvT1DE=xDmmLETu_J4ShipLs6Jfg1MVehYTqocWvsOyTQ8-FQ@mail.gmail.com>
	<CANpCgCMOwcqticEccUAKZARrsndQoCMvwMkQtpiBHN-XRJES6A@mail.gmail.com>
	<CABvT1DEucB-oZ8-ge1KoBCZu=oboJR9B0vaqBkHv3Nc-W1WT_g@mail.gmail.com>
Date: Thu, 4 Aug 2011 11:24:13 -0700
Message-ID: 
 <CANpCgCPEVcauL9NoUxDfXZ4uXe+wwaZffX3q0Hz_S2-RzWvZng@mail.gmail.com>
Subject: Re: unusable database in 1.0.1. steps to take?
From: Jason Konrad <jason@inboxq.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Couch will start up without any problems. I'm able to access other
databases normally. When requests start trying to interact with the
one problematic db that is when error messages like the one I provided
start to show up in the logs. There was maybe 10 req/sec trying to use
the problem db. After a couple min couch terminates.

On Thu, Aug 4, 2011 at 9:34 AM, Robert Newson <rnewson@apache.org> wrote:
> Hm, yeah, that might not be practical.
>
> Does the crash happen immediately on startup or do you need to query a
> document or a view first?
>
> B.
>
> On 4 August 2011 17:07, Jason Konrad <jason@inboxq.com> wrote:
>> The file system is ext3 and delayed_commits is set to false.
>>
>> I think its possible to share the file although its 65GB so I don't know=
 how
>> practical it would be.
>> On Aug 4, 2011 2:01 AM, "Robert Newson" <rnewson@apache.org> wrote:
>>> Hi Jason,
>>>
>>> What filesystem is this stored on? Are you running with
>>> delayed_commits set to true or false?
>>>
>>> Finally, are you able to share the database file with the CouchDB
>>> development team?
>>>
>>> B.
>>>
>>> On 4 August 2011 03:39, Jason Konrad <jason@inboxq.com> wrote:
>>>> I have a database that has become unusable today. The only way I can
>>>> get the couchdb server to function is to remove "thedb" which it
>>>> doesn't like. I've attached a file with some of the log data as well
>>>> as some errorr futon throws up. I'm looking for any ideas for next
>>>> steps to try and "recover" this database.
>>>>
>>>> This is running on a CentOS5 system with erlang-R12B-5.12.el5 and
>>>> couchdb-1.0.1-2.el5
>>>>
>>>> Let me give you a little context to how this came about. I have one
>>>> database, not the one with problems, that has a high update rate on
>>>> documents. Combined with smallish disks this requires compaction a
>>>> couple times a week. The compactions have been going well but the data
>>>> is not always freed up from the system after the compaction. In order
>>>> to get the disk space to free up I restart couchdb. This process had
>>>> been working for a few months but not today.
>>>>
>>>> On the restart of couchdb the first signs of something out of the
>>>> ordinary was happening, two views were being completely recalulated.
>>>> These calculations can take hours to finish but today that was not the
>>>> case. After about 30 min the views stopped calculating and couchdb
>>>> started throwing 500 response codes for everything no matter what
>>>> database was being used ( there are 26 total, 1 bad one ). I tried
>>>> restarting couchdb but all that would happen is the log file would get
>>>> loads of data like the stuff in the attached file and then eventually
>>>> couchdb would terminate.
>>>>
>>>>
>>>> Respect,
>>>> =A0Jason
>>>>
>>
>