Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
From: Marco Monteiro <marco@textovirtual.com>
Date: Fri, 11 May 2012 02:42:04 +0100
Message-ID: 
 <CADgE0Xinf+vs-KSGWtVDeJL0SxhuocCOriPb0MYt4LrcQZ+bAQ@mail.gmail.com>
Subject: CouchDB freezes
To: dev@couchdb.apache.org
Content-Type: multipart/alternative; boundary=e89a8f83a1dd2424cf04bfb8d95a

--e89a8f83a1dd2424cf04bfb8d95a
Content-Type: text/plain; charset=ISO-8859-1

Hello!

I'm running a load test on CouchDB. I have a cluster of 8 node.js servers
writing to
CouchDB. They write about 30000 documents per minute (500 per second).
There are
multiple concurrent requests form each server. There are no updates:
documents are
created and not modified.

I first tried CouchDB 1.1.1 from Debian 6.4 apt repo. After a few minutes,
CouchDB
starts freezing for a period of 1 to 3 seconds about every 10 seconds. It
keeps this
behaviour for some time and eventually it starts freezing more frequently
and for longer
periods. When the database has about 1.5 million documents, couchdb is
freezing for
more than 5 seconds each time.

I then tried CouchDB 1.2, from build-couch. The freezes happen with it
also, but the
behavior is much worse: in less than one minute it's freezing for 5 seconds
or more,
and it spends more time not doing anything than working.

When testing with 1.1.1 I was writing only to one database. With 1.2 I
tried with one database
and then with multiple databases but the problem was exactly the same.

The documents have about 10 properties, only numbers or string and the
strings are small
(about 20 chars each). The document IDs are generated in the app and have
the format

  <milliseconds since epoch>-<random 16 chars string>

When CouchDB freezes, it's processor use (from top) goes to zero. It does
not reply to read or write
requests. The disk does not seem to be the  problem as iostat reports near
0% utilization.
CPU is mostly idle, and from the 16 GB of RAM, some of it is free and is
not even used to
cache disk.

There are no error message in Couchdb log.

I tried this in two different machines and the problem is the same in both.

I did not change anything in the configuration files expect changing the
database dir to use
a RAID partition.

Anyone has any idea of what the problem could be?

Any help solving this issue is greatly appreciated.

Thanks,
Marco

--e89a8f83a1dd2424cf04bfb8d95a--