Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of jchris@gmail.com designates
 209.85.132.248 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=c5cyX4Kmof5HrXwCMj0LzRaNxFOMiuPBrtaGd/Se5fHt5Gx/YLbWBn+8v0NoxTxmHC
         HtzVt1Yxinc8wxIlAHB1BRMSbIVJlPvDsPpp1Df903WbJEk6XL8doaaXH/O+yNbQD6Rn
         inaqIrmYgjyeoq7g4WfTlEx8EFkBJC5VQs9Wo=
MIME-Version: 1.0
Sender: jchris@gmail.com
In-Reply-To: <d8cb87ee0905051043s66fbdf5ci350e2cbed9ba9278@mail.gmail.com>
References: <d8cb87ee0905040840i3929889fn86ce94018471ba3@mail.gmail.com>
	 <49FF5E3E.8020604@proven-corporation.com>
	 <d8cb87ee0905051043s66fbdf5ci350e2cbed9ba9278@mail.gmail.com>
Date: Tue, 5 May 2009 12:33:00 -0700
Message-ID: <e282921e0905051233o21e7302el7268c0fd8443cc48@mail.gmail.com>
Subject: Re: Insert performance
From: Chris Anderson <jchris@apache.org>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Tue, May 5, 2009 at 10:43 AM, Tom Nichols <tmnichols@gmail.com> wrote:
> So I did a rough calculation and it looks like I'm getting less than
> 1MB/s throughput in CouchDB --
>
> 3072 MB total / 6900 sec =3D 0.445 MB/s
>
> So if the disk throughput is ~20 to 30 MB/s then the bottleneck is
> somewhere in the database. =A0It's obviously not going to be anywhere
> close to raw disk I/O speeds but this still seems incredibly slow.
> Granted, I'm using a small instance... =A0I'll try a c1.medium and see
> if the results are drastically different.
>

Can you try running this benchmark script and see what you get for
insert performance:

http://gist.github.com/79279

> On Mon, May 4, 2009 at 5:29 PM, Jason Smith <jhs@proven-corporation.com> =
wrote:
>> Tom Nichols wrote:
>>>
>>> Hi, I have some questions about insert performance.
>>>
>>> I have a single CouchDB 0.9.0 node running on small EC2 instance. =A0I
>>> attached a huge EBS volume to it and mounted it where CouchDB's data
>>> files are stored. =A0I fired up about ruby scripts running inserts and
>>> after a weekend I only have about 30GB/ 12M rows of data... =A0Which
>>> seems small. =A0'top' tells me that my CPU is only about 30% utilized.
>>>
>>> Any idea what I might be doing wrong? =A0I pretty much just followed
>>> these instructions:
>>> http://wiki.apache.org/couchdb/Getting_started_with_Amazon_EC2
>>
>> Hi, Tom. =A0I believe I read somewhere before that the smallest EC2 inst=
ances
>> have a slower and/or higher-latency connection to EBS, so you might want=
 to
>> consider a large instance, or maybe even a high-memory small instance an=
d
>> see whether you get better "hardware" performance.
>>
>> Although strangely, when googling it, the first article I found says tha=
t
>> their benchmarks found no difference between EBS or even the ephemeral
>> filesystem.
>>
>> http://www.paessler.com/blog/2009/04/07/prtg-7/monitoring-cloud-performa=
nce-with-prtg-comparing-disk-speed-for-instance-stores-and-ebs-volumes-on-a=
mazon-ec2/
>>
>> On the other hand, here is a forum posting and a random benchmark indica=
ting
>> that more expensive instances get better throughput:
>>
>> http://developer.amazonwebservices.com/connect/message.jspa?messageID=3D=
125197
>> http://blog.getasysadmin.com/2009/02/mysql-benchmarks-using-amazon-ec2.h=
tml
>>
>> --
>> Jason Smith
>> Proven Corporation
>> Bangkok, Thailand
>> http://www.proven-corporation.com
>>
>


--=20
Chris Anderson
http://jchrisa.net
http://couch.io