incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <rnew...@apache.org>
Subject Re: CouchDB freezes
Date Fri, 11 May 2012 12:56:17 GMT
Perhaps CouchDB on this particular hardware isn't fast enough to cope
with 4,000 writes per second?

Does your node.js test send every update asynchronously or is it
carefully controlling qps? For what it's worth, I've benchmarked
successfully using a node.js library called nodeload
(https://github.com/benschmaus/nodeload). It's been a while since I
last used it, and node has changed a few dozen times since then, but
it was pretty solid and sane when I was using it.

B.

On 11 May 2012 13:48, Marco Monteiro <marco@textovirtual.com> wrote:
> Thanks, Robert.
>
> Disabling delayed commits did make the problem start later, but it is still
> there.
>
> It's funny that the first think that I checked when I first saw this
> problem was to
> make sure that delayed commits where enabled.
>
> Thanks,
> Marco
>
> On 11 May 2012 13:20, Robert Newson <rnewson@apache.org> wrote:
>
>> The first thing is to ensure you have disabled delayed commits;
>>
>> curl -XPUT -d '"false" localhost:5984/_config/couchdb/delayed_commits
>>
>> This is the production setting anyway (though not the default because
>> of complaints from incompetent benchmarkers). This will ensure an
>> fsync for each write and, as a consequence, will greatly smooth your
>> insert performance. Since you said you were inserting concurrently,
>> you should not experience a slowdown either.
>>
>> B.
>>
>> On 11 May 2012 02:42, Marco Monteiro <marco@textovirtual.com> wrote:
>> > Hello!
>> >
>> > I'm running a load test on CouchDB. I have a cluster of 8 node.js servers
>> > writing to
>> > CouchDB. They write about 30000 documents per minute (500 per second).
>> > There are
>> > multiple concurrent requests form each server. There are no updates:
>> > documents are
>> > created and not modified.
>> >
>> > I first tried CouchDB 1.1.1 from Debian 6.4 apt repo. After a few
>> minutes,
>> > CouchDB
>> > starts freezing for a period of 1 to 3 seconds about every 10 seconds. It
>> > keeps this
>> > behaviour for some time and eventually it starts freezing more frequently
>> > and for longer
>> > periods. When the database has about 1.5 million documents, couchdb is
>> > freezing for
>> > more than 5 seconds each time.
>> >
>> > I then tried CouchDB 1.2, from build-couch. The freezes happen with it
>> > also, but the
>> > behavior is much worse: in less than one minute it's freezing for 5
>> seconds
>> > or more,
>> > and it spends more time not doing anything than working.
>> >
>> > When testing with 1.1.1 I was writing only to one database. With 1.2 I
>> > tried with one database
>> > and then with multiple databases but the problem was exactly the same.
>> >
>> > The documents have about 10 properties, only numbers or string and the
>> > strings are small
>> > (about 20 chars each). The document IDs are generated in the app and have
>> > the format
>> >
>> >  <milliseconds since epoch>-<random 16 chars string>
>> >
>> > When CouchDB freezes, it's processor use (from top) goes to zero. It does
>> > not reply to read or write
>> > requests. The disk does not seem to be the  problem as iostat reports
>> near
>> > 0% utilization.
>> > CPU is mostly idle, and from the 16 GB of RAM, some of it is free and is
>> > not even used to
>> > cache disk.
>> >
>> > There are no error message in Couchdb log.
>> >
>> > I tried this in two different machines and the problem is the same in
>> both.
>> >
>> > I did not change anything in the configuration files expect changing the
>> > database dir to use
>> > a RAID partition.
>> >
>> > Anyone has any idea of what the problem could be?
>> >
>> > Any help solving this issue is greatly appreciated.
>> >
>> > Thanks,
>> > Marco
>>

Mime
View raw message