incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From João Ramos <joao.si...@gmail.com>
Subject Re: Error after creating hundred of views
Date Wed, 11 Jul 2012 20:17:34 GMT
Forget want I said: I really don't need that many views. If the key is [id,
year, month, day], I can filter like I want to using only one
view, although I have to call that view 10K times. I guess that's better
than calling 10K individual views.

But still, Couchdb shouldn't be having those problems with that number of
views, should it? There's nothing that limits the number of views a
database can / should have.

On Wed, Jul 11, 2012 at 5:44 PM, João Ramos <joao.sigma@gmail.com> wrote:

> But then how would I be able to get reductions for each id, filtered
> by whatever_you_were_previously_emitting_here?
>
>
> On Wed, Jul 11, 2012 at 3:40 PM, Keith Gable <ziggy@ignition-project.com>wrote:
>
>> Well I would emit a key of [id,
>> whatever_you_were_previously_emitting_here], it would be the same
>> functionally but I think would be easier to work with.
>> On Jul 11, 2012 4:36 AM, "João Ramos" <joao.sigma@gmail.com> wrote:
>>
>> > Well, it's the only way I figured I could solve my problem. I detailed
>> it
>> > in a previous thread named " Map/reduce problem"  (02 Jul). I ended up
>> > creating a view for each
>> > doc.type where the key is simply the data, and that means having lots of
>> > views. I didn't know of any limitations concerning the number of views a
>> > database can have.
>> >
>> > On Wed, Jul 11, 2012 at 1:31 AM, Keith Gable <
>> ziggy@ignition-project.com
>> > >wrote:
>> >
>> > > I would want to know why you're creating so many views. Views aren't
>> > > queries. I have maybe a dozen and my application is pretty large.
>> > > On Jul 10, 2012 7:15 PM, "João Ramos" <joao.sigma@gmail.com> wrote:
>> > >
>> > > > Something is still not right. I increased the filedescriptors to
>> 4096
>> > and
>> > > > the same thing keeps happening.
>> > > >
>> > > > I restarted Couchdb, and used _stats to check how many
>> filedescriptors
>> > > were
>> > > > open: about 10, so I proceeded in creating 1000 design docs and
>> > > > the filedescriptors count went up to about 1000. I inserted another
>> > 1000
>> > > > three more times and the count went up as excepted at around 4056.
>> > Then I
>> > > > queried those views (each design docs has only one view) and
>> everything
>> > > was
>> > > > fine. The 4000 queries were fast. To make sure I did them again and
>> > still
>> > > > no problem. Then I tried to insert another 1000 designs docs and the
>> > > error
>> > > > started showing up. I couldn't use the database for anything
>> anymore.
>> > > >
>> > > > Then I decided to compact the database and the filedescriptors
>> returned
>> > > > back to about 10. So far so good, so I started querying 1000 views
>> and
>> > > the
>> > > > count went up again. I managed to call up until 4000 different views
>> > > until
>> > > > the error reappeared.
>> > > >
>> > > > Does Couchdb ever closes the filedescriptors? Does limit on the
>> number
>> > > > of filedescriptors somehow influences the maximum number of design
>> > docs /
>> > > > views a database can have?
>> > > >
>> > > > On Mon, Jul 9, 2012 at 9:25 PM, Niels Boldt <nielsboldt@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Hi
>> > > > >
>> > > > > emfile means you run out of filedescriptors
>> > > > >
>> > > > > See  http://wiki.apache.org/couchdb/Performance#Resource_Limitsfor
>> > > > > further
>> > > > > information in increasing this limit
>> > > > >
>> > > > > Best Regards
>> > > > > Niels
>> > > > >
>> > > > > On Mon, Jul 9, 2012 at 8:01 PM, João Ramos <joao.sigma@gmail.com>
>> > > wrote:
>> > > > >
>> > > > > > I have a script that creates several hundred of views.
>> > > > > > At about 700, I start getting several errors and I've attached
>> the
>> > > log
>> > > > > > output of them.
>> > > > > >
>> > > > > > Thanks in advance.
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > BinaryConstructors ApS
>> > > > > Vestergade 10a, 4th
>> > > > > 1456 Kbh K
>> > > > > Denmark
>> > > > > phone: +4529722259
>> > > > > web: http://www.binaryconstructors.dk
>> > > > > mail: nb@binaryconstructors.dk
>> > > > > skype: nielsboldt
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message