Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.198.240 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=nuSP6I7RZvO3EK3IhcE0uTWYcRMz+yvzlh3XKMxGvuo9T1KGTMGUPb8yQu4sLcOO4A
         oHcwUGaN8qiTCeZkj8SkFSxXVOIHZaCfOCS1H8H56XacL0RNp32d2n7/I7YCVawhdtmo
         W940Lpyk+TwLwypeByRjbQA16q072N9G0PV3E=
Message-ID: <e2111bbb0809111505p5666d025sc1bee9baa78ce86b@mail.gmail.com>
Date: Thu, 11 Sep 2008 18:05:33 -0400
From: "Paul Davis" <paul.joseph.davis@gmail.com>
To: couchdb-user@incubator.apache.org
Subject: Re: view access keeps timing out
In-Reply-To: <928bdd8e0809111432o4030c855t144c506c44648b91@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <928bdd8e0809101925q1f5963ccsca8e0425a9b289eb@mail.gmail.com>
	 <e282921e0809102208v567bbe9ek3ab521c627850e97@mail.gmail.com>
	 <928bdd8e0809111011y7851e58flb60d691b4826e5d5@mail.gmail.com>
	 <7c40ded80809111044h694a671cta793be9a161ef575@mail.gmail.com>
	 <928bdd8e0809111231t4c17f7e3q9eec1d44bee99140@mail.gmail.com>
	 <e2111bbb0809111244t528c3b79h879f10c339cfeda0@mail.gmail.com>
	 <928bdd8e0809111432o4030c855t144c506c44648b91@mail.gmail.com>

1. Database looks not too big to cause problems
2. The couchjs process won't start until its first needed. And it'll
stick around after it has been spawned for reuse. Not popping up when
accessing the view is bad.
3. Good.
4. Haven't groked the complete workings of the view generation, but
hazarding a guess, it might not get created until the first doc is
returned from the view process.

The URL vs auto generated ID could be the culprit here. When you
submit docs, are you making sure to properly escape the urls? Does
futon work to view the database? (alternatively, can you get docs from
http://localhost:5984/db_name/_all_docs?count=1)

Next on the list:

1. Change the log level to debug in your couch.ini and see if anything pops up.

Paul

On Thu, Sep 11, 2008 at 5:32 PM, william kinney
<william.kinney@gmail.com> wrote:
> 1. about 10KB each. futon shows "16325 documents, total size 181.0 MB".
> 2. Ahh looks like after i do a fresh boot of couchdb and access one of the
> views, I never see:
> /usr/local/lib/couchdb/bin/couchjs /usr/local/share/couchdb/server/main.js
> When couchdb starts up, there is:
> /bin/sh -e /usr/local/bin/couchdb -c /usr/local/etc/couchdb/couch.ini -b -r
> 5.......
> /bin/sh -e /usr/local/bin/couchdb -c /usr/local/etc/couchdb/couch.ini -b -r
> 5.....
> /usr/lib/erlang/erts-5.5.2/bin/beam.smp -Bd -- ......
>
> no couchjs (which I think is normal). When I access a different database's
> view however, I see the couchjs process. Which seems to stick around even
> after the view data has been returned (normal?). However, the couchjs
> process never pops up if I access one of the views of the large database.
> 3. No havn't touched any of the documents after the timeout. Only attemping
> to acces a view.
> 4. It exists. for the working databases, i see two "rck"s and then some
> binary content". For the large/non-working database, all I see is the
> "rck"s, I assume this means the views havn't been successfully compiled.
>
> So it looks like the issue is that it is failing when trying to spawn the
> couchjs process for the view. no?
>
> One thing to note that is different about this database is that for a large
> amount of documents, it is using a URL (from a web crawl) for the ID,
> instead of the autogenerated one.
>
> Thanks for the help,
> Will
>
> On Thu, Sep 11, 2008 at 3:44 PM, Paul Davis <paul.joseph.davis@gmail.com>wrote:
>
>> Things to check,
>>
>> 1. How big are your docs?
>> 2. Is there a javascript map process that's running even after the
>> view times out?
>> 3. Are you touching things even after a timeout?
>> 4. Check to make sure there's a design directory in the couchdb
>> directory. These files should be in ".db_name_design"
>>
>> For 3, if you get a time out, does it timeout a second time? Make sure
>> you don't touch anything though. Whenever your design doc changes the
>> entire view has to be rebuilt.
>>
>> Paul
>>
>> On Thu, Sep 11, 2008 at 3:31 PM, william kinney
>> <william.kinney@gmail.com> wrote:
>> > I simplified the map function to:
>> >
>> > function(doc) {
>> >  emit(doc._id, doc);
>> > }
>> >
>> > and with count=3&update=false it is still timing out. I'm not sure what
>> is
>> > going on. I forgot to mention, the server is a vmware image...but I don't
>> > think that has anything do it with it (unless the disk io is being
>> mangled
>> > by vmware?).
>> >
>> > I have another database of 5000 records, and it's not having an issue in
>> > accessing some of the views, even after some updates. Could it be
>> something
>> > in the data that is making the views fail?
>> >
>> > The only difference I can see between the two databases is that the one
>> that
>> > is failing I created the views after inserting all 10k documents, the 5k
>> it
>> > was before and somewhat incrementally updated as it grew.
>> >
>> > I suppose I'll try populating another database of 10k docs and see what
>> > happens.
>> >
>> > I wish there was a way I could see how the view is progressing, other
>> than
>> > requesting and waiting.
>> >
>> > Thanks,
>> > Will
>> >
>> > On Thu, Sep 11, 2008 at 1:44 PM, Jeremy Wall <jwall@google.com> wrote:
>> >
>> >> are you trying to return all of the documents at once? does it improve
>> if
>> >> you restrict the results with a count=100 or similar? I can possibly see
>> a
>> >> scenario where it might time out just trying to server several thousand
>> >> documents to you at once. CouchDB doesn't exactly stream the documents
>> to
>> >> you one at a time like a sql database might.
>> >>
>> >> On Thu, Sep 11, 2008 at 12:11 PM, william kinney
>> >> <william.kinney@gmail.com>wrote:
>> >>
>> >> > thanks for those! I was googling trying to find example reduce
>> functions,
>> >> > in
>> >> > vain.
>> >> >
>> >> > It seems my view (w/ reduce function removed) is still timing out.
>> >> > curl: (52) Empty reply from server
>> >> >
>> >> > real    85m15.699s
>> >> >
>> >> > I'm not sure what's going on. The map function isn't that complicated.
>> >> Any
>> >> > ideas on how I could find out what's going on?
>> >> >
>> >> > Thanks,
>> >> > Will
>> >> >
>> >> >
>> >> > On Thu, Sep 11, 2008 at 1:08 AM, Chris Anderson <jchris@grabb.it>
>> wrote:
>> >> >
>> >> > > On Wed, Sep 10, 2008 at 7:25 PM, william kinney
>> >> > > <william.kinney@gmail.com> wrote:
>> >> > > > reduce function:
>> >> > > > function(keys, values) {
>> >> > > >  return values;
>> >> > > > }
>> >> > > >
>> >> > >
>> >> > > The problem is with your reduce function (and the existing
>> >> > > documentation for reduce, which should make this clear, but
>> doesn't).
>> >> > >
>> >> > > Reduce should only be used to emit scalar values. In your case, you
>> >> > > would be better not to define a reduce function at all, and just
>> query
>> >> > > the map with startkey and endkey ranges. For example reduce
>> functions,
>> >> > > see couch_tests.js's reduce test here:
>> >> > >
>> >> > >
>> >> > >
>> >> >
>> >>
>> http://svn.apache.org/repos/asf/incubator/couchdb/trunk/share/www/script/couch_tests.js
>> >> > >
>> >> > > There are also some examples in various blog posts I've written
>> about
>> >> > > reduce:
>> >> > >
>> >> > > Google search for those posts: http://tinyurl.com/couchdb-reduce
>> >> > >
>> >> > > Hope this helps!
>> >> > >
>> >> > > Chris
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > > --
>> >> > > Chris Anderson
>> >> > > http://jchris.mfdz.com
>> >> > >
>> >> >
>> >>
>> >
>>
>