Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of jchris@gmail.com designates
 209.85.222.182 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=tjgTmPqvIX1FJWRbDsyFqe4Rk08ckAiqmnNQk3KY/hTF8kK3nOtR+TDol/lxenRemN
         3G9+1uvxOWl8/nObAcPWJ9HFCsCHVkIlO4qiNeZuN6ZeDN+PF/B3kjznjVVhofes/mFH
         tw7XZhe6+6qOveQ2lb4L9uxEklXSvCwSXEoYg=
MIME-Version: 1.0
Sender: jchris@gmail.com
In-Reply-To: <C061C036-D30B-4F42-9DC9-571A80866E98@gmail.com>
References: <C061C036-D30B-4F42-9DC9-571A80866E98@gmail.com>
Date: Tue, 1 Dec 2009 08:30:57 -0800
Message-ID: <e282921e0912010830p47761c4tc7542813ce6ac0d5@mail.gmail.com>
Subject: Re: CouchDB Hangs on View Creation
From: Chris Anderson <jchris@apache.org>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Tue, Dec 1, 2009 at 6:26 AM, Luke Sneeringer
<lukesneeringer@gmail.com> wrote:
>> > Onto my issue. I have a database with about 59,000 documents. Some con=
tain attachments,
>> some do not. I tried to replicate (pull, not push) said database from on=
e CouchDB instance
>> to another one. The replication failed several times inexplicably, both =
using curl on the
>> command line (Ubuntu 9.10) and Futon. Finally, it succeeded in Futon. I =
am defining "failed"
>> here as that replication seemed to hang; the number of documents in the =
target database did
>> not increase, was less than the number of documents in the source databa=
se (by many thousands),
>> and the item disappeared from the status page in Futon. Also, on the fai=
led attempts, replication
>> failed in different places each time.
>> >
>>
>> What is your couchdb version on both ends of the replication?
> I'm using 0.10.0 on both sides.
>
>>
>> Also, pay attention to database file size on the target. Sometimes
>> with big attachments the doc count is unchanged as binary data is
>> written for a long time.
> I actually started watching the binary file size on view creation, and it=
 would stop increasing as well. I did not, however, monitor it on the repli=
cation itself.
>>
>> > So eventually it succeeded. Huzzah. Except now I can't create any view=
s. It just doesn't
>> work. I never see my results. So, I went and started looking at the file=
 being created in
>> /var/lib/couchdb/0.10.0/.dbname_design/. There's a .view file that gets =
created, and its size
>> goes up for awhile and then stops going up. At the time that file size c=
eases to increase,
>> I get a erl_crash.dump file that is 0 bytes in length. Not really valuab=
le.
>> >
>> > The view scripts themselves are "known good"; they work on the source =
server.
>>
>> Maybe a difference in versions. Can you supply a link to the view script=
s?
> I'm using 0.10.0 in both cases.
> I attempted a couple of different views, but this was the first one and t=
he one I was repeatedly working on:


> function(doc) {
> =A0 =A0for (var i in doc.groups) {
> =A0 =A0 =A0 =A0if (i =3D=3D=3D 'author') {
> =A0 =A0 =A0 =A0 =A0 =A0for (var j in doc.groups[i]) {
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0emit([doc.groups[i][j], doc.timestamp], do=
c)
> =A0 =A0 =A0 =A0 =A0 =A0}
> =A0 =A0 =A0 =A0}
> =A0 =A0}
> }

Do you have some gigantic docs, or are most of them less than 4kB?

Also, this looks like an equivalent function to me (but shouldn't make
a difference unless you have very large docs):

function(doc) {
  if (doc.groups && doc.groups.author) {
    doc.groups.author.forEach(function(a) {
      emit([a, doc.timestamp], null)
    });
  }
}

the main difference here being one less nested loop. The other
difference is that I'm emitting null instead of doc as the value. This
would be a good diagnostic to see how much faster it with smaller
values. Also, you are sure you don't have any reduces elsewhere in the
design document? All views in a ddoc are indexed together, so if you
have a different view working with this one, it could be the culprit.

> And thank you, by the way!
>
> Best Regards,
> Luke Sneeringer


--=20
Chris Anderson
http://jchrisa.net
http://couch.io