Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Date: Mon, 9 Nov 2009 11:07:21 -0800
From: James Marca <jmarca@translab.its.uci.edu>
To: user@couchdb.apache.org
Subject: compaction quirkiness
Message-ID: <20091109190721.GC27049@translab.its.uci.edu>
Mail-Followup-To: user@couchdb.apache.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.4.1i

Hi All,

I'm running a recent svn checkout of couchdb (actually a git pull from
halorgium / couchdb). (1) 

I have twelve large databases that I am populating with data (one db
per month of raw data...manual "sharding").

As each data-loading perl job finishes, it sends off a call to compact
the database as writing is done.

Unfortunately, multiple compaction jobs seem to crash couchdb, which
cancels the compaction (but doesn't seem to hurt the data loading
processes).  I'm not sure why the server crashes, as I can't find
anything significant in my log files. All I see is:

...

 [Fri, 06 Nov 2009 18:28:32 GMT] [info] [<0.7156.2>] 127.0.0.1 - -
 'PUT' /d12_2007_04morehash/1209013_2007-04-29T03:00:00 201

 [Fri, 06 Nov 2009 18:28:38 GMT] [info] [<0.31.0>] Apache CouchDB has
 started on http://127.0.0.1:5984/

 [Fri, 06 Nov 2009 18:28:39 GMT] [info] [<0.100.0>] 127.0.0.1 - - 'PUT'
 /d12_2007_04morehash/1214006_2007-04-29T06:00:00 201

...

I have logging on info, not debug, but putting logging on debug didn't
appear to give more helpful information.  I'd expect a crash condition
to spit out some sort of "error" level log message anyway.

If I run just one compaction at a time, all is well.  Also, restarting
the crashed compactions restarts them at the right place (it picks up
where it left off).  So the problem isn't devastating by any means,
but it seems like a bug that multiple compactions crash the server.

I'm willing to upgrade to a newer git pull, or try different things to
try to isolate the problem, short of mucking with the source code (I
do not know erlang).

Details of install:

CouchDB: {"couchdb":"Welcome","version":"0.11.0b1e2a54d1-git"}
(last commit dated Oct 29 in the logs)

Erlang: Erlang R13B02 (erts-5.7.3) [source] [64-bit] [smp:8:8] [rq:8]
[async-threads:0] [kernel-poll:false]

OS: Gentoo Linux amd64 10.0/no-multilib  (reasonably up to date, gcc
version 4.3.4 in case that matters to erlang/couchdb)


Regards,
James Marca

(1) Yes, I'm aware I can expect things to break running a git version
of CouchDB.  I'm not complaining, rather just hoping my problem can
lead to finding and fixing a bug.


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.