couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Davis" <paul.joseph.da...@gmail.com>
Subject Re: view access keeps timing out
Date Fri, 12 Sep 2008 00:27:20 GMT
That's pretty fucking weird. No two ways about it.

Your doicds sound fine if futon can show them all right. Obviously the
docs are in there correctly and what not with _all_docs working.

The CPU pinging to 90% for 90 minutes is consistent with what you'd
expect for generating a view on 10K docs. Did you say the couchjs
process is running for this or no? It seems awfully weird that its
doing something with no error and not creating those files. I'm pretty
certain they should appear as soon as the map process starts. You
might try nuking the .db_name_design directory and trying again.

Other than that, I'm fresh out of ideas. If something occurs to me
I'll ping back. Maybe when Jan or Noah wake up in a couple hours
they'll see something I'm missing.

Paul


On Thu, Sep 11, 2008 at 8:21 PM, william kinney
<william.kinney@gmail.com> wrote:
> I can access /db_name/_all_docs?count=1 fine, actually /db_name/_all_docs is
> fine and super quick. The URLs as IDs I can see are all properly stored and
> returned.
>
> When storing, I made sure to use URLEncoder(String, "utf-8") (Java).
>
> Perhaps could it be the javascript faulting on the map functions, failing on
> the URL/ID somehow?
>  scratch that, just tested a view function that doesn't emit a URL/ID,
> still fails. maybe it's still accessed in processing the view since it is an
> ID?
>
> What is also weird is that beam.smp spikes to 100% after first requesting
> the view, and doesn't return to normal for about 90 minutes.
>
> Everything seems fine except the view it seems.
>
> the log, from turning on debug, has:
> [debug] [<0.49.0>] Spawning new update process for view group
> _design/standard in database news.
> [debug] [<0.73.0>] Reseting group index "_design/standard" in db news
>
> and nothing else useful I think (other than HTTP headers).
>
> Thanks for continuing to help!
> Will
>
>
> On Thu, Sep 11, 2008 at 6:05 PM, Paul Davis <paul.joseph.davis@gmail.com>wrote:
>
>> 1. Database looks not too big to cause problems
>> 2. The couchjs process won't start until its first needed. And it'll
>> stick around after it has been spawned for reuse. Not popping up when
>> accessing the view is bad.
>> 3. Good.
>> 4. Haven't groked the complete workings of the view generation, but
>> hazarding a guess, it might not get created until the first doc is
>> returned from the view process.
>>
>> The URL vs auto generated ID could be the culprit here. When you
>> submit docs, are you making sure to properly escape the urls? Does
>> futon work to view the database? (alternatively, can you get docs from
>> http://localhost:5984/db_name/_all_docs?count=1)
>>
>> Next on the list:
>>
>> 1. Change the log level to debug in your couch.ini and see if anything pops
>> up.
>>
>> Paul
>>
>> On Thu, Sep 11, 2008 at 5:32 PM, william kinney
>> <william.kinney@gmail.com> wrote:
>> > 1. about 10KB each. futon shows "16325 documents, total size 181.0 MB".
>> > 2. Ahh looks like after i do a fresh boot of couchdb and access one of
>> the
>> > views, I never see:
>> > /usr/local/lib/couchdb/bin/couchjs
>> /usr/local/share/couchdb/server/main.js
>> > When couchdb starts up, there is:
>> > /bin/sh -e /usr/local/bin/couchdb -c /usr/local/etc/couchdb/couch.ini -b
>> -r
>> > 5.......
>> > /bin/sh -e /usr/local/bin/couchdb -c /usr/local/etc/couchdb/couch.ini -b
>> -r
>> > 5.....
>> > /usr/lib/erlang/erts-5.5.2/bin/beam.smp -Bd -- ......
>> >
>> > no couchjs (which I think is normal). When I access a different
>> database's
>> > view however, I see the couchjs process. Which seems to stick around even
>> > after the view data has been returned (normal?). However, the couchjs
>> > process never pops up if I access one of the views of the large database.
>> > 3. No havn't touched any of the documents after the timeout. Only
>> attemping
>> > to acces a view.
>> > 4. It exists. for the working databases, i see two "rck"s and then some
>> > binary content". For the large/non-working database, all I see is the
>> > "rck"s, I assume this means the views havn't been successfully compiled.
>> >
>> > So it looks like the issue is that it is failing when trying to spawn the
>> > couchjs process for the view. no?
>> >
>> > One thing to note that is different about this database is that for a
>> large
>> > amount of documents, it is using a URL (from a web crawl) for the ID,
>> > instead of the autogenerated one.
>> >
>> > Thanks for the help,
>> > Will
>> >
>> > On Thu, Sep 11, 2008 at 3:44 PM, Paul Davis <paul.joseph.davis@gmail.com
>> >wrote:
>> >
>> >> Things to check,
>> >>
>> >> 1. How big are your docs?
>> >> 2. Is there a javascript map process that's running even after the
>> >> view times out?
>> >> 3. Are you touching things even after a timeout?
>> >> 4. Check to make sure there's a design directory in the couchdb
>> >> directory. These files should be in ".db_name_design"
>> >>
>> >> For 3, if you get a time out, does it timeout a second time? Make sure
>> >> you don't touch anything though. Whenever your design doc changes the
>> >> entire view has to be rebuilt.
>> >>
>> >> Paul
>> >>
>> >> On Thu, Sep 11, 2008 at 3:31 PM, william kinney
>> >> <william.kinney@gmail.com> wrote:
>> >> > I simplified the map function to:
>> >> >
>> >> > function(doc) {
>> >> >  emit(doc._id, doc);
>> >> > }
>> >> >
>> >> > and with count=3&update=false it is still timing out. I'm not sure
>> what
>> >> is
>> >> > going on. I forgot to mention, the server is a vmware image...but I
>> don't
>> >> > think that has anything do it with it (unless the disk io is being
>> >> mangled
>> >> > by vmware?).
>> >> >
>> >> > I have another database of 5000 records, and it's not having an issue
>> in
>> >> > accessing some of the views, even after some updates. Could it be
>> >> something
>> >> > in the data that is making the views fail?
>> >> >
>> >> > The only difference I can see between the two databases is that the
>> one
>> >> that
>> >> > is failing I created the views after inserting all 10k documents, the
>> 5k
>> >> it
>> >> > was before and somewhat incrementally updated as it grew.
>> >> >
>> >> > I suppose I'll try populating another database of 10k docs and see
>> what
>> >> > happens.
>> >> >
>> >> > I wish there was a way I could see how the view is progressing, other
>> >> than
>> >> > requesting and waiting.
>> >> >
>> >> > Thanks,
>> >> > Will
>> >> >
>> >> > On Thu, Sep 11, 2008 at 1:44 PM, Jeremy Wall <jwall@google.com>
>> wrote:
>> >> >
>> >> >> are you trying to return all of the documents at once? does it
>> improve
>> >> if
>> >> >> you restrict the results with a count=100 or similar? I can possibly
>> see
>> >> a
>> >> >> scenario where it might time out just trying to server several
>> thousand
>> >> >> documents to you at once. CouchDB doesn't exactly stream the
>> documents
>> >> to
>> >> >> you one at a time like a sql database might.
>> >> >>
>> >> >> On Thu, Sep 11, 2008 at 12:11 PM, william kinney
>> >> >> <william.kinney@gmail.com>wrote:
>> >> >>
>> >> >> > thanks for those! I was googling trying to find example reduce
>> >> functions,
>> >> >> > in
>> >> >> > vain.
>> >> >> >
>> >> >> > It seems my view (w/ reduce function removed) is still timing
out.
>> >> >> > curl: (52) Empty reply from server
>> >> >> >
>> >> >> > real    85m15.699s
>> >> >> >
>> >> >> > I'm not sure what's going on. The map function isn't that
>> complicated.
>> >> >> Any
>> >> >> > ideas on how I could find out what's going on?
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Will
>> >> >> >
>> >> >> >
>> >> >> > On Thu, Sep 11, 2008 at 1:08 AM, Chris Anderson <jchris@grabb.it>
>> >> wrote:
>> >> >> >
>> >> >> > > On Wed, Sep 10, 2008 at 7:25 PM, william kinney
>> >> >> > > <william.kinney@gmail.com> wrote:
>> >> >> > > > reduce function:
>> >> >> > > > function(keys, values) {
>> >> >> > > >  return values;
>> >> >> > > > }
>> >> >> > > >
>> >> >> > >
>> >> >> > > The problem is with your reduce function (and the existing
>> >> >> > > documentation for reduce, which should make this clear,
but
>> >> doesn't).
>> >> >> > >
>> >> >> > > Reduce should only be used to emit scalar values. In
your case,
>> you
>> >> >> > > would be better not to define a reduce function at all,
and just
>> >> query
>> >> >> > > the map with startkey and endkey ranges. For example
reduce
>> >> functions,
>> >> >> > > see couch_tests.js's reduce test here:
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> http://svn.apache.org/repos/asf/incubator/couchdb/trunk/share/www/script/couch_tests.js
>> >> >> > >
>> >> >> > > There are also some examples in various blog posts I've
written
>> >> about
>> >> >> > > reduce:
>> >> >> > >
>> >> >> > > Google search for those posts: http://tinyurl.com/couchdb-reduce
>> >> >> > >
>> >> >> > > Hope this helps!
>> >> >> > >
>> >> >> > > Chris
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> > > --
>> >> >> > > Chris Anderson
>> >> >> > > http://jchris.mfdz.com
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >
>>
>

Mime
View raw message