incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Insert performance
Date Wed, 06 May 2009 14:28:36 GMT
Hi Tom, those numbers seems to be in the right ballpark to me.  The  
single-core / dual-core distinction isn't quite right for EC2, where  
it's all about "compute units".  Regardless of the actual CPUs used, a  
c1.medium should get 5x the cycles of an m1.small.  Amazon's  
hypervisor will "steal" cycles from your m1.small instance to preserve  
this property if it has to.

The fact that your wallclock time dropped by 4.3x when you upgraded to  
a c1.medium suggests that you were mostly CPU bound.  Cheers,

Adam

On May 6, 2009, at 9:39 AM, Tom Nichols wrote:

> Hi Chris, here are my results:
> m1.small running Ubuntu Intrepid:
> real    1m3.049s
> user    0m25.040s
> sys     0m2.240s
>
> c1.medium instance:
> real    0m14.752s
> user    0m23.990s
> sys     0m2.220s
>
> In practice, the c1.medium seemed to run my ruby scripts a bit faster,
> giving a thoughput of about 0.98 MB/s, or 730 docs/s.  I thought the
> m1.small instance was still dual-core; if it's a single-core machine
> then that at least partly explains why running a bunch of load scripts
> on the same box as CouchDB would be so slow.
>
> Thoughts?  Is this the performance I should expect?
>
> Thanks.
> -Tom
>
>
> On Tue, May 5, 2009 at 3:33 PM, Chris Anderson <jchris@apache.org>  
> wrote:
>> On Tue, May 5, 2009 at 10:43 AM, Tom Nichols <tmnichols@gmail.com>  
>> wrote:
>>> So I did a rough calculation and it looks like I'm getting less than
>>> 1MB/s throughput in CouchDB --
>>>
>>> 3072 MB total / 6900 sec = 0.445 MB/s
>>>
>>> So if the disk throughput is ~20 to 30 MB/s then the bottleneck is
>>> somewhere in the database.  It's obviously not going to be anywhere
>>> close to raw disk I/O speeds but this still seems incredibly slow.
>>> Granted, I'm using a small instance...  I'll try a c1.medium and see
>>> if the results are drastically different.
>>>
>>
>> Can you try running this benchmark script and see what you get for
>> insert performance:
>>
>> http://gist.github.com/79279
>>
>>> On Mon, May 4, 2009 at 5:29 PM, Jason Smith <jhs@proven-corporation.com 
>>> > wrote:
>>>> Tom Nichols wrote:
>>>>>
>>>>> Hi, I have some questions about insert performance.
>>>>>
>>>>> I have a single CouchDB 0.9.0 node running on small EC2  
>>>>> instance.  I
>>>>> attached a huge EBS volume to it and mounted it where CouchDB's  
>>>>> data
>>>>> files are stored.  I fired up about ruby scripts running inserts  
>>>>> and
>>>>> after a weekend I only have about 30GB/ 12M rows of data...  Which
>>>>> seems small.  'top' tells me that my CPU is only about 30%  
>>>>> utilized.
>>>>>
>>>>> Any idea what I might be doing wrong?  I pretty much just followed
>>>>> these instructions:
>>>>> http://wiki.apache.org/couchdb/Getting_started_with_Amazon_EC2
>>>>
>>>> Hi, Tom.  I believe I read somewhere before that the smallest EC2  
>>>> instances
>>>> have a slower and/or higher-latency connection to EBS, so you  
>>>> might want to
>>>> consider a large instance, or maybe even a high-memory small  
>>>> instance and
>>>> see whether you get better "hardware" performance.
>>>>
>>>> Although strangely, when googling it, the first article I found  
>>>> says that
>>>> their benchmarks found no difference between EBS or even the  
>>>> ephemeral
>>>> filesystem.
>>>>
>>>> http://www.paessler.com/blog/2009/04/07/prtg-7/monitoring-cloud-performance-with-prtg-comparing-disk-speed-for-instance-stores-and-ebs-volumes-on-amazon-ec2/
>>>>
>>>> On the other hand, here is a forum posting and a random benchmark  
>>>> indicating
>>>> that more expensive instances get better throughput:
>>>>
>>>> http://developer.amazonwebservices.com/connect/message.jspa?messageID=125197
>>>> http://blog.getasysadmin.com/2009/02/mysql-benchmarks-using-amazon-ec2.html
>>>>
>>>> --
>>>> Jason Smith
>>>> Proven Corporation
>>>> Bangkok, Thailand
>>>> http://www.proven-corporation.com
>>>>
>>>
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchrisa.net
>> http://couch.io
>>


Mime
View raw message