couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brad Schick <schi...@gmail.com>
Subject Re: Bad write performance
Date Tue, 17 Jun 2008 21:56:07 GMT
On 06/17/2008 12:57 PM, Brian Whitman wrote:
>
> On Jun 17, 2008, at 3:11 PM, Brad Schick wrote:
>
>> I am seeing very poor write performance running CouchDB on Ubuntu Linux
>> 8.04. I built CouchDB from source about a week ago, and I am using the
>> Python wrapper to access it all on localhost.
>>
>> I am trying to add over 90,000 documents (each is about 400 bytes) and
>> finding that I can only add about 16 documents per second. And while
>> this is happening, my CPU is about 75% idle.
>
> Are you using the update method in python (which uses _bulk_docs) ?
> You need the svn version of the python wrapper for it.
>
> On a slow machine (dual proc 1.8 GHz) using the python wrapper I added
> 100K docs 10K at a time, each with a random string, uuid, float and
> int in 180 seconds (roughly 550 documents/second.) One CPU was pegged
> at 100% with beam.smp during the update and the python client did not
> even rate on top during the update (it was just waiting for a response.)
>
> I couldn't increase my chunk size much past 10K -- couch would return
> a (very long) error if I tried 5 20K chunks for example.
>

I have the code from svn, but I haven't tried bulk uploading yet because
I've been primarily testing schema.py which currently only does single
document stores. Maybe I'll hook that up to bulk writes and see how it goes.

-Brad

Mime
View raw message