couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "william kinney" <william.kin...@gmail.com>
Subject Re: view access keeps timing out
Date Fri, 12 Sep 2008 01:27:51 GMT
correct, the couchjs is not running.
I just tried removing .db_name_design directory from
/usr/local/var/lib/couchdb and trying again, same thing.

Thanks again,
Will


On Thu, Sep 11, 2008 at 8:27 PM, Paul Davis <paul.joseph.davis@gmail.com>wrote:

> That's pretty fucking weird. No two ways about it.
>
> Your doicds sound fine if futon can show them all right. Obviously the
> docs are in there correctly and what not with _all_docs working.
>
> The CPU pinging to 90% for 90 minutes is consistent with what you'd
> expect for generating a view on 10K docs. Did you say the couchjs
> process is running for this or no? It seems awfully weird that its
> doing something with no error and not creating those files. I'm pretty
> certain they should appear as soon as the map process starts. You
> might try nuking the .db_name_design directory and trying again.
>
> Other than that, I'm fresh out of ideas. If something occurs to me
> I'll ping back. Maybe when Jan or Noah wake up in a couple hours
> they'll see something I'm missing.
>
> Paul
>
>
> On Thu, Sep 11, 2008 at 8:21 PM, william kinney
> <william.kinney@gmail.com> wrote:
> > I can access /db_name/_all_docs?count=1 fine, actually /db_name/_all_docs
> is
> > fine and super quick. The URLs as IDs I can see are all properly stored
> and
> > returned.
> >
> > When storing, I made sure to use URLEncoder(String, "utf-8") (Java).
> >
> > Perhaps could it be the javascript faulting on the map functions, failing
> on
> > the URL/ID somehow?
> >  scratch that, just tested a view function that doesn't emit a URL/ID,
> > still fails. maybe it's still accessed in processing the view since it is
> an
> > ID?
> >
> > What is also weird is that beam.smp spikes to 100% after first requesting
> > the view, and doesn't return to normal for about 90 minutes.
> >
> > Everything seems fine except the view it seems.
> >
> > the log, from turning on debug, has:
> > [debug] [<0.49.0>] Spawning new update process for view group
> > _design/standard in database news.
> > [debug] [<0.73.0>] Reseting group index "_design/standard" in db news
> >
> > and nothing else useful I think (other than HTTP headers).
> >
> > Thanks for continuing to help!
> > Will
> >
> >
> > On Thu, Sep 11, 2008 at 6:05 PM, Paul Davis <paul.joseph.davis@gmail.com
> >wrote:
> >
> >> 1. Database looks not too big to cause problems
> >> 2. The couchjs process won't start until its first needed. And it'll
> >> stick around after it has been spawned for reuse. Not popping up when
> >> accessing the view is bad.
> >> 3. Good.
> >> 4. Haven't groked the complete workings of the view generation, but
> >> hazarding a guess, it might not get created until the first doc is
> >> returned from the view process.
> >>
> >> The URL vs auto generated ID could be the culprit here. When you
> >> submit docs, are you making sure to properly escape the urls? Does
> >> futon work to view the database? (alternatively, can you get docs from
> >> http://localhost:5984/db_name/_all_docs?count=1)
> >>
> >> Next on the list:
> >>
> >> 1. Change the log level to debug in your couch.ini and see if anything
> pops
> >> up.
> >>
> >> Paul
> >>
> >> On Thu, Sep 11, 2008 at 5:32 PM, william kinney
> >> <william.kinney@gmail.com> wrote:
> >> > 1. about 10KB each. futon shows "16325 documents, total size 181.0
> MB".
> >> > 2. Ahh looks like after i do a fresh boot of couchdb and access one of
> >> the
> >> > views, I never see:
> >> > /usr/local/lib/couchdb/bin/couchjs
> >> /usr/local/share/couchdb/server/main.js
> >> > When couchdb starts up, there is:
> >> > /bin/sh -e /usr/local/bin/couchdb -c /usr/local/etc/couchdb/couch.ini
> -b
> >> -r
> >> > 5.......
> >> > /bin/sh -e /usr/local/bin/couchdb -c /usr/local/etc/couchdb/couch.ini
> -b
> >> -r
> >> > 5.....
> >> > /usr/lib/erlang/erts-5.5.2/bin/beam.smp -Bd -- ......
> >> >
> >> > no couchjs (which I think is normal). When I access a different
> >> database's
> >> > view however, I see the couchjs process. Which seems to stick around
> even
> >> > after the view data has been returned (normal?). However, the couchjs
> >> > process never pops up if I access one of the views of the large
> database.
> >> > 3. No havn't touched any of the documents after the timeout. Only
> >> attemping
> >> > to acces a view.
> >> > 4. It exists. for the working databases, i see two "rck"s and then
> some
> >> > binary content". For the large/non-working database, all I see is the
> >> > "rck"s, I assume this means the views havn't been successfully
> compiled.
> >> >
> >> > So it looks like the issue is that it is failing when trying to spawn
> the
> >> > couchjs process for the view. no?
> >> >
> >> > One thing to note that is different about this database is that for a
> >> large
> >> > amount of documents, it is using a URL (from a web crawl) for the ID,
> >> > instead of the autogenerated one.
> >> >
> >> > Thanks for the help,
> >> > Will
> >> >
> >> > On Thu, Sep 11, 2008 at 3:44 PM, Paul Davis <
> paul.joseph.davis@gmail.com
> >> >wrote:
> >> >
> >> >> Things to check,
> >> >>
> >> >> 1. How big are your docs?
> >> >> 2. Is there a javascript map process that's running even after the
> >> >> view times out?
> >> >> 3. Are you touching things even after a timeout?
> >> >> 4. Check to make sure there's a design directory in the couchdb
> >> >> directory. These files should be in ".db_name_design"
> >> >>
> >> >> For 3, if you get a time out, does it timeout a second time? Make
> sure
> >> >> you don't touch anything though. Whenever your design doc changes the
> >> >> entire view has to be rebuilt.
> >> >>
> >> >> Paul
> >> >>
> >> >> On Thu, Sep 11, 2008 at 3:31 PM, william kinney
> >> >> <william.kinney@gmail.com> wrote:
> >> >> > I simplified the map function to:
> >> >> >
> >> >> > function(doc) {
> >> >> >  emit(doc._id, doc);
> >> >> > }
> >> >> >
> >> >> > and with count=3&update=false it is still timing out. I'm
not sure
> >> what
> >> >> is
> >> >> > going on. I forgot to mention, the server is a vmware image...but
I
> >> don't
> >> >> > think that has anything do it with it (unless the disk io is being
> >> >> mangled
> >> >> > by vmware?).
> >> >> >
> >> >> > I have another database of 5000 records, and it's not having an
> issue
> >> in
> >> >> > accessing some of the views, even after some updates. Could it
be
> >> >> something
> >> >> > in the data that is making the views fail?
> >> >> >
> >> >> > The only difference I can see between the two databases is that
the
> >> one
> >> >> that
> >> >> > is failing I created the views after inserting all 10k documents,
> the
> >> 5k
> >> >> it
> >> >> > was before and somewhat incrementally updated as it grew.
> >> >> >
> >> >> > I suppose I'll try populating another database of 10k docs and
see
> >> what
> >> >> > happens.
> >> >> >
> >> >> > I wish there was a way I could see how the view is progressing,
> other
> >> >> than
> >> >> > requesting and waiting.
> >> >> >
> >> >> > Thanks,
> >> >> > Will
> >> >> >
> >> >> > On Thu, Sep 11, 2008 at 1:44 PM, Jeremy Wall <jwall@google.com>
> >> wrote:
> >> >> >
> >> >> >> are you trying to return all of the documents at once? does
it
> >> improve
> >> >> if
> >> >> >> you restrict the results with a count=100 or similar? I can
> possibly
> >> see
> >> >> a
> >> >> >> scenario where it might time out just trying to server several
> >> thousand
> >> >> >> documents to you at once. CouchDB doesn't exactly stream the
> >> documents
> >> >> to
> >> >> >> you one at a time like a sql database might.
> >> >> >>
> >> >> >> On Thu, Sep 11, 2008 at 12:11 PM, william kinney
> >> >> >> <william.kinney@gmail.com>wrote:
> >> >> >>
> >> >> >> > thanks for those! I was googling trying to find example
reduce
> >> >> functions,
> >> >> >> > in
> >> >> >> > vain.
> >> >> >> >
> >> >> >> > It seems my view (w/ reduce function removed) is still
timing
> out.
> >> >> >> > curl: (52) Empty reply from server
> >> >> >> >
> >> >> >> > real    85m15.699s
> >> >> >> >
> >> >> >> > I'm not sure what's going on. The map function isn't
that
> >> complicated.
> >> >> >> Any
> >> >> >> > ideas on how I could find out what's going on?
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> > Will
> >> >> >> >
> >> >> >> >
> >> >> >> > On Thu, Sep 11, 2008 at 1:08 AM, Chris Anderson <
> jchris@grabb.it>
> >> >> wrote:
> >> >> >> >
> >> >> >> > > On Wed, Sep 10, 2008 at 7:25 PM, william kinney
> >> >> >> > > <william.kinney@gmail.com> wrote:
> >> >> >> > > > reduce function:
> >> >> >> > > > function(keys, values) {
> >> >> >> > > >  return values;
> >> >> >> > > > }
> >> >> >> > > >
> >> >> >> > >
> >> >> >> > > The problem is with your reduce function (and the
existing
> >> >> >> > > documentation for reduce, which should make this
clear, but
> >> >> doesn't).
> >> >> >> > >
> >> >> >> > > Reduce should only be used to emit scalar values.
In your
> case,
> >> you
> >> >> >> > > would be better not to define a reduce function
at all, and
> just
> >> >> query
> >> >> >> > > the map with startkey and endkey ranges. For example
reduce
> >> >> functions,
> >> >> >> > > see couch_tests.js's reduce test here:
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >>
> >>
> http://svn.apache.org/repos/asf/incubator/couchdb/trunk/share/www/script/couch_tests.js
> >> >> >> > >
> >> >> >> > > There are also some examples in various blog posts
I've
> written
> >> >> about
> >> >> >> > > reduce:
> >> >> >> > >
> >> >> >> > > Google search for those posts:
> http://tinyurl.com/couchdb-reduce
> >> >> >> > >
> >> >> >> > > Hope this helps!
> >> >> >> > >
> >> >> >> > > Chris
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > >
> >> >> >> > > --
> >> >> >> > > Chris Anderson
> >> >> >> > > http://jchris.mfdz.com
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >> >
> >> >>
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message