Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 78A8A9260 for ; Fri, 23 Mar 2012 11:35:13 +0000 (UTC) Received: (qmail 54886 invoked by uid 500); 23 Mar 2012 11:35:12 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 54849 invoked by uid 500); 23 Mar 2012 11:35:11 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 54841 invoked by uid 99); 23 Mar 2012 11:35:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Mar 2012 11:35:11 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.215.180] (HELO mail-ey0-f180.google.com) (209.85.215.180) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Mar 2012 11:35:03 +0000 Received: by eaal12 with SMTP id l12so944468eaa.11 for ; Fri, 23 Mar 2012 04:34:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:content-transfer-encoding :x-gm-message-state; bh=WYuTwoGQkMbptbxF7BldyY5oE8Q7uc4bi43jKiNv1MY=; b=ZR98HR1751dhlqBWIIV+nZ4UVTtlG1oBQojHS7WRIx/lf/1NgjOogEyfhOuWEtSHJK 7ckgJuzvkZRWNtnqWOzGI0CPFbSOmz20s6WHeqJW12Cg5r/egnF+7efvPsJCzhDgf6Bi x1aHn/cSsxcsUEXKEfpgoCOsFpy6RMPlPNpuF5UWJOYnug4xQWdDUSpagitMt/N+oJdv qA0nT3c8C2o3F8W2KQnWwAO3c553pcCu0XjdzVfrRvH7MuOYP5PPiEWvr9ihXJrASEFm 5Ml/MKLumCt4I+93SGYUM8+RU3/zThqwIxtFOPr8IOdts2dEBDMOQOqhsApeo0lxpM8Z lfJQ== MIME-Version: 1.0 Received: by 10.213.15.129 with SMTP id k1mr874382eba.70.1332502483062; Fri, 23 Mar 2012 04:34:43 -0700 (PDT) Received: by 10.213.2.211 with HTTP; Fri, 23 Mar 2012 04:34:42 -0700 (PDT) X-Originating-IP: [81.23.57.130] In-Reply-To: References: Date: Fri, 23 Mar 2012 11:34:42 +0000 Message-ID: Subject: Re: Why are reads from CouchDB so slow? (1.5MB/s or thereabouts) From: Jonathan Williamson To: user@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQk7xARFchedc56RmTovqMg4Z/FvQlmnZ45pKaYUIzX3efLbJ+uzA3CtovZX+3uBBz8/SD2L X-Virus-Checked: Checked by ClamAV on apache.org # time curl http://localhost:5984/ {"couchdb":"Welcome","version":"1.1.1"} real 0m0.012s user 0m0.000s sys 0m0.000s On Fri, Mar 23, 2012 at 11:31 AM, Jason Smith wrote: > Useful as well would be the response time for: > > =A0 =A0curl http://localhost:5984/ > > To get a baseline for CouchDB's raw HTTP response time, with no i/o. > > On Fri, Mar 23, 2012 at 11:21 AM, CGS wrote: >> ...also, it would be nice to know how you inserted the documents and if >> they have attachments (at that size, I suppose they have). Could you als= o >> provide a result from ping 127.0.0.1 for response time fluctuations? >> >> CGS >> >> >> >> >> On Fri, Mar 23, 2012 at 12:16 PM, Jason Smith wrote: >> >>> Hi, Jonathan. >>> >>> Can we start by identifying how fast you think **is** reasonable. What >>> speeds were you expecting? And on what grounds do you base that >>> expectation? >>> >>> (For example, perhaps connecting and fetching one row from MySQL?) >>> >>> On Fri, Mar 23, 2012 at 11:06 AM, Jonathan Williamson >>> wrote: >>> > Hi everyone, >>> > >>> > I posted this question on Stack Overflow and Jason Smith suggested I >>> > try this group instead. >>> > >>> >> I have a CouchDB (1.1.1) server running that contains a lot of >>> documents in the 400-600KB size range. >>> >> >>> >> If I time fetching a full document from the database (not from a vie= w, >>> just the raw document) it takes 200-400ms to complete which equates to >>> around 1.5MB/s throughput. >>> >> >>> >> If I write the same data to raw files on disk they load in 10-20ms >>> (around 25-50 MB/s). >>> >> >>> >> I'd expect CouchDB to have some overhead, but an order of magnitude >>> (and some) seems crazy for what is essentially a read! >>> >> >>> >> Can anyone shed some light onto why this might be the case? >>> >> >>> >> Update: As requested below, a timing from curl: >>> >> >>> >> # time curl http://localhost:5984/[dbname]/[documentname] >>> >> >>> >> real =A0 =A00m0.684s >>> >> user =A0 =A00m0.004s >>> >> sys =A0 =A0 0m0.020s >>> >> >>> >> The fetched document was 642842 bytes. I've tested it on both a >>> standard 1TB harddisk and an EC2 instance (EBS volume) with similar res= ults. >>> > >>> > There was some discussion about the post which can be found here: >>> > >>> http://stackoverflow.com/questions/9807200/why-are-reads-from-couchdb-s= o-slow-1-5mb-s-or-thereabouts >>> > >>> > Thanks, >>> > >>> > Jon. >>> >>> >>> >>> -- >>> Iris Couch >>> > > > > -- > Iris Couch --=20 Jon Jonathan Williamson Systems Architect ::::::::::::::::::::::::::::::::::::::::::: +44 (0) 7843 301 246 =A0mobile +44 (0) 208 242 6364 =A0switch www.netcopy.co.uk :::::::::::::::::::::::::::::::::::::::::: IMPORTANT NOTICE: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Netcopy Limited is a Company registered in England and Wales under Company No. 07308350 Registered office: 2 East Common, Harpenden, Herts, AL5 1BJ The contents of this email (and any attachment): (1) are confidential and may be legally privileged - if it is not meant for you, please tell the sender, do not forward or copy the contents and delete it from your system immediately; (2) come from its author and may not necessarily reflect the opinions of Netcopy Ltd. While emails and attachments are virus checked, we cannot accept any liability in respect of any viruses. We may monitor emails sent to Netcopy. Want to know more about Netcopy? Visit our website: www.netcopy.co.uk