couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dustin Whitney <dustin.whit...@gmail.com>
Subject Re: Insert performance
Date Tue, 05 May 2009 17:05:57 GMT
That's kind of a bummer with respect to the small EC2 instances and
throughput.  What can the CouchDB guys share about their setup with CouchIO
- I'm not sure if I have the name right, but I mean the service where you
guys host the CouchDB on EC2.

-Dustin

On Tue, May 5, 2009 at 12:59 PM, Tom Nichols <tmnichols@gmail.com> wrote:

> Thanks Jason.
>
> I removed the EBS volume and things seemed a little faster.  CouchDB
> is actually saturating the CPU now, so it looks like I'm getting
> "accurate" benchmarks now.  i.e. I'm actually exercising CouchDB and
> I'm not limited by some hardware/OS latency.
>
> So I had 10 concurrent ruby processes performing batch inserts into
> two test databases.  After 2 hours, I have 1.1M documents (1.5 GB) in
> each database, for a total of 2GB and 2.2 million documents.  Does
> this still seem slow?
>
> Thanks.
> -Tom
>
>
> On Mon, May 4, 2009 at 5:29 PM, Jason Smith <jhs@proven-corporation.com>
> wrote:
> > Tom Nichols wrote:
> >>
> >> Hi, I have some questions about insert performance.
> >>
> >> I have a single CouchDB 0.9.0 node running on small EC2 instance.  I
> >> attached a huge EBS volume to it and mounted it where CouchDB's data
> >> files are stored.  I fired up about ruby scripts running inserts and
> >> after a weekend I only have about 30GB/ 12M rows of data...  Which
> >> seems small.  'top' tells me that my CPU is only about 30% utilized.
> >>
> >> Any idea what I might be doing wrong?  I pretty much just followed
> >> these instructions:
> >> http://wiki.apache.org/couchdb/Getting_started_with_Amazon_EC2
> >
> > Hi, Tom.  I believe I read somewhere before that the smallest EC2
> instances
> > have a slower and/or higher-latency connection to EBS, so you might want
> to
> > consider a large instance, or maybe even a high-memory small instance and
> > see whether you get better "hardware" performance.
> >
> > Although strangely, when googling it, the first article I found says that
> > their benchmarks found no difference between EBS or even the ephemeral
> > filesystem.
> >
> >
> http://www.paessler.com/blog/2009/04/07/prtg-7/monitoring-cloud-performance-with-prtg-comparing-disk-speed-for-instance-stores-and-ebs-volumes-on-amazon-ec2/
> >
> > On the other hand, here is a forum posting and a random benchmark
> indicating
> > that more expensive instances get better throughput:
> >
> >
> http://developer.amazonwebservices.com/connect/message.jspa?messageID=125197
> >
> http://blog.getasysadmin.com/2009/02/mysql-benchmarks-using-amazon-ec2.html
> >
> > --
> > Jason Smith
> > Proven Corporation
> > Bangkok, Thailand
> > http://www.proven-corporation.com
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message