couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Volker Mische <volker.mis...@gmail.com>
Subject Re: Why are reads from CouchDB so slow? (1.5MB/s or thereabouts)
Date Fri, 23 Mar 2012 13:17:26 GMT
I agree that using a trusted benchmarking tool is the way to go. Tough
what Jonathan sees is pretty clear. It's the JSON -> Eterm -> String
conversion that CouchDB is currently doing. Filipe proposed a patch that
store the raw JSON on disk, to get rid most of this conversion. I don't
remember exactly, but I'm pretty sure he provided sensible benchmarking
results back then.

Hence the point of this thread shouldn't be: go, do it properly. But: we
have a clue why it is so slow, we don't store raw JSON, but Eterms on
disk, that need to be assembled to a string everytime you request it.

Cheers,
  Volker

On 03/23/2012 01:44 PM, Robert Newson wrote:
> I can confirm that you won't get nginx's sendfile level of
> performance, if that helps at all. However, if we're to investigate a
> performance issue I want to be sure that an issue has been competently
> discovered. The minimum for me would be seeing results from a trusted
> benchmarking tool.
> 
> B.
> 
> On 23 March 2012 12:41, Jonathan Williamson <jon@netcopy.co.uk> wrote:
>> I'm aware of that - but it still doesn't explain the disparity. I was
>> using curl for the Nginx tests too...
>>
>> Is the reason you're asking the question because you believe the
>> performance should be much better than I'm seeing?
>>
>> On Fri, Mar 23, 2012 at 12:39 PM, Robert Newson <rnewson@apache.org> wrote:
>>> curl is not a benchmarking tool. Each curl command is opening a new
>>> tcp connection, etc. Can you try with httperf, apachebench, tsung,
>>> nodeload?
>>>
>>> B.
>>>
>>> On 23 March 2012 12:36, Jonathan Williamson <jon@netcopy.co.uk> wrote:
>>>> It was just a script that looped over the keys for 1000 documents,
>>>> fetching them one after another (using curl).
>>>>
>>>> On Fri, Mar 23, 2012 at 12:31 PM, Robert Newson <rnewson@apache.org>
wrote:
>>>>> Using which http benchmarking tool (and exactly which settings)?
>>>>>
>>>>> On 23 March 2012 12:30, Jonathan Williamson <jon@netcopy.co.uk>
wrote:
>>>>>> My initial CouchDB test was over 1000 GETs (all to roughly similar
>>>>>> documents), taking an average of the total time vs the total amount
of
>>>>>> data recieved.
>>>>>>
>>>>>> On Fri, Mar 23, 2012 at 12:26 PM, Robert Newson <rnewson@apache.org>
wrote:
>>>>>>> Are you really extrapolating mb/s from a single curl command?
>>
>>
>>
>> --
>> Jon
>>
>> Jonathan Williamson
>> Systems Architect
>> :::::::::::::::::::::::::::::::::::::::::::
>> +44 (0) 7843 301 246  mobile
>> +44 (0) 208 242 6364  switch
>>
>> www.netcopy.co.uk
>> ::::::::::::::::::::::::::::::::::::::::::
>>
>> IMPORTANT NOTICE:
>> ========================================================================================
>> Netcopy Limited is a Company registered in England and Wales under
>> Company No. 07308350
>> Registered office: 2 East Common, Harpenden, Herts, AL5 1BJ
>> The contents of this email (and any attachment):
>> (1) are confidential and may be legally privileged - if it is not
>> meant for you, please tell the sender, do not forward or copy the
>> contents and delete it from your system immediately;
>> (2) come from its author and may not necessarily reflect the opinions
>> of Netcopy Ltd.
>> While emails and attachments are virus checked, we cannot accept any
>> liability in respect of any viruses.
>> We may monitor emails sent to Netcopy.
>> Want to know more about Netcopy?
>> Visit our website: www.netcopy.co.uk


Mime
View raw message