incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Gable <zi...@ignition-project.com>
Subject Re: Error after creating hundred of views
Date Wed, 11 Jul 2012 20:33:09 GMT
Well, don't forget you could emit things twice either. So if you wanted to
be able to get [id, year, month, day] and [year, month, day] there's no
reason you couldn't do this. Of course, you want to disambiguate [year,
month, day], so probably you would want [-1, year, month, day] or something
like that.

I agree that CouchDB shouldn't be choking on that many views, but if
CouchDB wants one file for each view, you could totally run out of file
descriptors even if you cranked it up to the max if you're talking about
10K file descriptors and your system only allows 32,767 or something like
that.



---
Keith Gable
A+ Certified Professional
Network+ Certified Professional
Storage+ Certified Professional
Mobile Application Developer / Web Developer



On Wed, Jul 11, 2012 at 3:17 PM, João Ramos <joao.sigma@gmail.com> wrote:

> Forget want I said: I really don't need that many views. If the key is [id,
> year, month, day], I can filter like I want to using only one
> view, although I have to call that view 10K times. I guess that's better
> than calling 10K individual views.
>
> But still, Couchdb shouldn't be having those problems with that number of
> views, should it? There's nothing that limits the number of views a
> database can / should have.
>
> On Wed, Jul 11, 2012 at 5:44 PM, João Ramos <joao.sigma@gmail.com> wrote:
>
> > But then how would I be able to get reductions for each id, filtered
> > by whatever_you_were_previously_emitting_here?
> >
> >
> > On Wed, Jul 11, 2012 at 3:40 PM, Keith Gable <ziggy@ignition-project.com
> >wrote:
> >
> >> Well I would emit a key of [id,
> >> whatever_you_were_previously_emitting_here], it would be the same
> >> functionally but I think would be easier to work with.
> >> On Jul 11, 2012 4:36 AM, "João Ramos" <joao.sigma@gmail.com> wrote:
> >>
> >> > Well, it's the only way I figured I could solve my problem. I detailed
> >> it
> >> > in a previous thread named " Map/reduce problem"  (02 Jul). I ended up
> >> > creating a view for each
> >> > doc.type where the key is simply the data, and that means having lots
> of
> >> > views. I didn't know of any limitations concerning the number of
> views a
> >> > database can have.
> >> >
> >> > On Wed, Jul 11, 2012 at 1:31 AM, Keith Gable <
> >> ziggy@ignition-project.com
> >> > >wrote:
> >> >
> >> > > I would want to know why you're creating so many views. Views aren't
> >> > > queries. I have maybe a dozen and my application is pretty large.
> >> > > On Jul 10, 2012 7:15 PM, "João Ramos" <joao.sigma@gmail.com>
wrote:
> >> > >
> >> > > > Something is still not right. I increased the filedescriptors
to
> >> 4096
> >> > and
> >> > > > the same thing keeps happening.
> >> > > >
> >> > > > I restarted Couchdb, and used _stats to check how many
> >> filedescriptors
> >> > > were
> >> > > > open: about 10, so I proceeded in creating 1000 design docs and
> >> > > > the filedescriptors count went up to about 1000. I inserted
> another
> >> > 1000
> >> > > > three more times and the count went up as excepted at around
4056.
> >> > Then I
> >> > > > queried those views (each design docs has only one view) and
> >> everything
> >> > > was
> >> > > > fine. The 4000 queries were fast. To make sure I did them again
> and
> >> > still
> >> > > > no problem. Then I tried to insert another 1000 designs docs
and
> the
> >> > > error
> >> > > > started showing up. I couldn't use the database for anything
> >> anymore.
> >> > > >
> >> > > > Then I decided to compact the database and the filedescriptors
> >> returned
> >> > > > back to about 10. So far so good, so I started querying 1000
views
> >> and
> >> > > the
> >> > > > count went up again. I managed to call up until 4000 different
> views
> >> > > until
> >> > > > the error reappeared.
> >> > > >
> >> > > > Does Couchdb ever closes the filedescriptors? Does limit on the
> >> number
> >> > > > of filedescriptors somehow influences the maximum number of design
> >> > docs /
> >> > > > views a database can have?
> >> > > >
> >> > > > On Mon, Jul 9, 2012 at 9:25 PM, Niels Boldt <nielsboldt@gmail.com
> >
> >> > > wrote:
> >> > > >
> >> > > > > Hi
> >> > > > >
> >> > > > > emfile means you run out of filedescriptors
> >> > > > >
> >> > > > > See
> http://wiki.apache.org/couchdb/Performance#Resource_Limitsfor
> >> > > > > further
> >> > > > > information in increasing this limit
> >> > > > >
> >> > > > > Best Regards
> >> > > > > Niels
> >> > > > >
> >> > > > > On Mon, Jul 9, 2012 at 8:01 PM, João Ramos <
> joao.sigma@gmail.com>
> >> > > wrote:
> >> > > > >
> >> > > > > > I have a script that creates several hundred of views.
> >> > > > > > At about 700, I start getting several errors and I've
attached
> >> the
> >> > > log
> >> > > > > > output of them.
> >> > > > > >
> >> > > > > > Thanks in advance.
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > BinaryConstructors ApS
> >> > > > > Vestergade 10a, 4th
> >> > > > > 1456 Kbh K
> >> > > > > Denmark
> >> > > > > phone: +4529722259
> >> > > > > web: http://www.binaryconstructors.dk
> >> > > > > mail: nb@binaryconstructors.dk
> >> > > > > skype: nielsboldt
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message