couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <jch...@apache.org>
Subject Re: CouchDB Hangs on View Creation
Date Tue, 01 Dec 2009 16:30:57 GMT
On Tue, Dec 1, 2009 at 6:26 AM, Luke Sneeringer
<lukesneeringer@gmail.com> wrote:
>> > Onto my issue. I have a database with about 59,000 documents. Some contain attachments,
>> some do not. I tried to replicate (pull, not push) said database from one CouchDB
instance
>> to another one. The replication failed several times inexplicably, both using curl
on the
>> command line (Ubuntu 9.10) and Futon. Finally, it succeeded in Futon. I am defining
"failed"
>> here as that replication seemed to hang; the number of documents in the target database
did
>> not increase, was less than the number of documents in the source database (by many
thousands),
>> and the item disappeared from the status page in Futon. Also, on the failed attempts,
replication
>> failed in different places each time.
>> >
>>
>> What is your couchdb version on both ends of the replication?
> I'm using 0.10.0 on both sides.
>
>>
>> Also, pay attention to database file size on the target. Sometimes
>> with big attachments the doc count is unchanged as binary data is
>> written for a long time.
> I actually started watching the binary file size on view creation, and it would stop
increasing as well. I did not, however, monitor it on the replication itself.
>>
>> > So eventually it succeeded. Huzzah. Except now I can't create any views. It
just doesn't
>> work. I never see my results. So, I went and started looking at the file being created
in
>> /var/lib/couchdb/0.10.0/.dbname_design/. There's a .view file that gets created,
and its size
>> goes up for awhile and then stops going up. At the time that file size ceases to
increase,
>> I get a erl_crash.dump file that is 0 bytes in length. Not really valuable.
>> >
>> > The view scripts themselves are "known good"; they work on the source server.
>>
>> Maybe a difference in versions. Can you supply a link to the view scripts?
> I'm using 0.10.0 in both cases.
> I attempted a couple of different views, but this was the first one and the one I was
repeatedly working on:


> function(doc) {
>    for (var i in doc.groups) {
>        if (i === 'author') {
>            for (var j in doc.groups[i]) {
>                emit([doc.groups[i][j], doc.timestamp], doc)
>            }
>        }
>    }
> }

Do you have some gigantic docs, or are most of them less than 4kB?

Also, this looks like an equivalent function to me (but shouldn't make
a difference unless you have very large docs):

function(doc) {
  if (doc.groups && doc.groups.author) {
    doc.groups.author.forEach(function(a) {
      emit([a, doc.timestamp], null)
    });
  }
}

the main difference here being one less nested loop. The other
difference is that I'm emitting null instead of doc as the value. This
would be a good diagnostic to see how much faster it with smaller
values. Also, you are sure you don't have any reduces elsewhere in the
design document? All views in a ddoc are indexed together, so if you
have a different view working with this one, it could be the culprit.

> And thank you, by the way!
>
> Best Regards,
> Luke Sneeringer



-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Mime
View raw message