From user-return-20167-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Fri Mar 23 11:39:03 2012 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A4DC2970B for ; Fri, 23 Mar 2012 11:39:03 +0000 (UTC) Received: (qmail 70763 invoked by uid 500); 23 Mar 2012 11:39:02 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 70727 invoked by uid 500); 23 Mar 2012 11:39:02 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 70716 invoked by uid 99); 23 Mar 2012 11:39:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Mar 2012 11:39:02 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [74.125.83.52] (HELO mail-ee0-f52.google.com) (74.125.83.52) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Mar 2012 11:38:55 +0000 Received: by eekd4 with SMTP id d4so964531eek.11 for ; Fri, 23 Mar 2012 04:38:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding :x-gm-message-state; bh=LrmBGt3IxMKdxJ/RnAnpem0krg1rJL9JU7Gai+oG7HM=; b=Hc3WIIwx4kEmnYRpsZuAEex+90xwKXCQxt6X99W3RxCe9W46Eo1/qbE4qSL5bjRxUX ZKouDKCITtyQW7t1LDvMLEYd0cY+JHTTscobaptBkXKd7xU+fv3F/83zAkaJf2SQRiJR fzTDZSBhOuvvv1oNir1M752k+qXNrqI3/BV6GmK2D9u9i2gg8sbFBZgLHA/4sl+Mk42j 1FtNSBlO2x4v56LTTEIBhKkLwkzMWwfPobcLP2s6Xd186zRmLRO2GgMMOs+4vnzH0GS1 GrhSpEjhGj7HbIhp5wjuI9S1I8a5EYsSvz3JrRUO7NNprR5LlU+B6pyZSnnZAXsGLD+e Dbmw== MIME-Version: 1.0 Received: by 10.213.8.79 with SMTP id g15mr885380ebg.108.1332502714350; Fri, 23 Mar 2012 04:38:34 -0700 (PDT) Received: by 10.213.2.211 with HTTP; Fri, 23 Mar 2012 04:38:34 -0700 (PDT) X-Originating-IP: [81.23.57.130] In-Reply-To: References: Date: Fri, 23 Mar 2012 11:38:34 +0000 Message-ID: Subject: Re: Why are reads from CouchDB so slow? (1.5MB/s or thereabouts) From: Jonathan Williamson To: user@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQn3WoD8d+yTkKYlbu77FlfWzYlkmwwFi7vQne7gMmDUUN7PdbZqTzcnUFVVcqXAdNvlNUtH X-Virus-Checked: Checked by ClamAV on apache.org The documents were inserted via the HTTP API. There are no attachments, just (largish and quite complex) documents. On Fri, Mar 23, 2012 at 11:21 AM, CGS wrote: > ...also, it would be nice to know how you inserted the documents and if > they have attachments (at that size, I suppose they have). Could you also > provide a result from ping 127.0.0.1 for response time fluctuations? > > CGS > > > > > On Fri, Mar 23, 2012 at 12:16 PM, Jason Smith wrote: > >> Hi, Jonathan. >> >> Can we start by identifying how fast you think **is** reasonable. What >> speeds were you expecting? And on what grounds do you base that >> expectation? >> >> (For example, perhaps connecting and fetching one row from MySQL?) >> >> On Fri, Mar 23, 2012 at 11:06 AM, Jonathan Williamson >> wrote: >> > Hi everyone, >> > >> > I posted this question on Stack Overflow and Jason Smith suggested I >> > try this group instead. >> > >> >> I have a CouchDB (1.1.1) server running that contains a lot of >> documents in the 400-600KB size range. >> >> >> >> If I time fetching a full document from the database (not from a view= , >> just the raw document) it takes 200-400ms to complete which equates to >> around 1.5MB/s throughput. >> >> >> >> If I write the same data to raw files on disk they load in 10-20ms >> (around 25-50 MB/s). >> >> >> >> I'd expect CouchDB to have some overhead, but an order of magnitude >> (and some) seems crazy for what is essentially a read! >> >> >> >> Can anyone shed some light onto why this might be the case? >> >> >> >> Update: As requested below, a timing from curl: >> >> >> >> # time curl http://localhost:5984/[dbname]/[documentname] >> >> >> >> real =A0 =A00m0.684s >> >> user =A0 =A00m0.004s >> >> sys =A0 =A0 0m0.020s >> >> >> >> The fetched document was 642842 bytes. I've tested it on both a >> standard 1TB harddisk and an EC2 instance (EBS volume) with similar resu= lts. >> > >> > There was some discussion about the post which can be found here: >> > >> http://stackoverflow.com/questions/9807200/why-are-reads-from-couchdb-so= -slow-1-5mb-s-or-thereabouts >> > >> > Thanks, >> > >> > Jon. >> >> >> >> -- >> Iris Couch >>