Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of adam.kocoloski@gmail.com
 designates 209.85.221.179 as permitted sender)
Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes
Mime-Version: 1.0 (Apple Message framework v1076)
Subject: Re: couchdb slowed down
From: Adam Kocoloski <kocolosk@apache.org>
In-Reply-To: <fbbaf7b0910061418y17151f7ej5269b7b95e04f8b6@mail.gmail.com>
Date: Tue, 6 Oct 2009 21:19:20 -0400
Content-Transfer-Encoding: 7bit
Message-Id: <ED76EBD4-C275-4F6F-9C70-F7B654FD4D54@apache.org>
References: <fbbaf7b0910061418y17151f7ej5269b7b95e04f8b6@mail.gmail.com>
To: user@couchdb.apache.org

On Oct 6, 2009, at 5:18 PM, Norman Barker wrote:

> I have upgraded couchdb from 0.9 to the trunk and the document write
> speed has really slowed down.
>
> My sequence is to call an erlang function get_docs_revs (my code) to
> find if the document exists as follows
>
> get_doc_revs(Db, DocId) ->
>  case DocId of
>    undefined ->
>      Id = couch_util:new_uuid(),
>      Revs = {0, []};
>    _ ->
>      case couch_db:get_doc_info(Db, DocId) of
>        {ok, Info} ->
>           Id = Info#doc_info.id,
>           [{rev_info, {Start, Rev}, _seq, _deleted, _body_sp}|_] =
> Info#doc_info.revs,
>           Revs = {Start, [Rev]};
>        _ ->
>          Id = DocId,
>          Revs = {0, []}
>      end
>  end,
>  {Id, Revs}.
>
> and then to write this doc to couch by calling write_doc_to_couch(Db,
> #doc{id=Id, revs=Revs, body={JsonObj}}, []) where this is defined as;
>
> write_doc_to_couch(Db, Doc, Options) ->
>   case couch_db:update_doc(Db,  Doc, Options) of
>      {ok, NewRev} ->
>         {Doc#doc.id, NewRev};
>      Error ->
>         throw({error, Error})
>   end.
>
> With this sequence I was writing about 500 docs in 5 seconds with 0.9,
> now it has slowed down to 30 seconds with trunk, my JSON generating
> code hasn't changed at all, the only thing that has had to change
> since moving from 0.9 to trunk is the get_docs_revs above - has this
> become a bottle neck in couchdb?
>
> My temporary work around is to spawn the writes, but this still
> doesn't really solve why it is has slowed down so much. Is there a
> 'cheaper' way of getting the current revision?
>
> thanks,
>
> Norman

Hi Norman, you might check the value for delayed_commits in the  
[couchdb] section of the .ini files. If set to false your serial write  
performance will be slower because CouchDB will fsync before returning  
from update_doc/3.  If set to true CouchDB will fsync at most once per  
second.  It's a tradeoff between durability and throughput.  For a  
while delayed_commits was set to false on trunk.  Best,

Adam