incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nils Breunese <N.Breun...@vpro.nl>
Subject RE: external handlers are not in sync with commits
Date Tue, 27 Jul 2010 21:48:03 GMT
We ran into performance problems when we put a site that was pretty couchdb-lucene heavy into
production. The query times in couchdb-lucene were fast, but we believe it was all those concurrent
queries that killed performance, since they all had to go through this single externals pipe
to couchdb-lucene. Using the _changes feed as couchdb-lucene's input sounds like a good idea,
but then couchdb-lucene queries also need to be going directly to the couchdb-lucene instance,
right? I believe it supports that now, but applications may need to be modified for this.
Or is there a way that this could still go through the current _fti URL's (without adding
some mod_rewrite like magic).

Nils.
________________________________________
Van: Robert Newson [robert.newson@gmail.com]
Verzonden: dinsdag 27 juli 2010 23:41
Aan: user@couchdb.apache.org
Onderwerp: Re: external handlers are not in sync with commits

reading _changes instead of using the (deprecated?) externals feature
would avoid the problem?

B.

On Tue, Jul 27, 2010 at 10:37 PM, J Chris Anderson <jchris@apache.org> wrote:
>
> On Jul 27, 2010, at 2:30 PM, Norman Barker wrote:
>
>> Hi,
>>
>> I have written couchdb-clucene
>> (http://github.com/normanb/couchdb-clucene) and am doing a lot of
>> testing with heavy datasets where I am sending a bulk doc request with
>> 10 docs at a time, a couple of these every second for a couple of
>> minutes.
>>
>> Very quickly couchdb backs up and hogs the cpu since the database
>> commit and return doesn't wait for an external handler to do its job.
>> The model of fire and forget is fine and I like it, very similar to
>> JMS, however since the external process is a singleton it has to be
>> very quick to keep up with load or the system slowly backs up.
>>
>> Is there a way to either define a pool of externals, or to change the
>> default behaviour from fire and forget?
>>
>
> Yes, but involves feature development for CouchDB. Essentially we need the externals
protocol to be non-blocking. There is a thread on dev@ that touches on this. I'm not sure
who wants to own making the patch, but the technical requirements are pretty well known.
>
> Thank you for working on something so awesome!
>
> Chris
>
>> thanks,
>>
>> Norman
>
>

De informatie vervat in deze  e-mail en meegezonden bijlagen is uitsluitend bedoeld voor gebruik
door de geadresseerde en kan vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
verspreiding en/of verstrekking van deze informatie aan derden is voorbehouden aan geadresseerde.
De VPRO staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden
e-mail, noch voor tijdige ontvangst daarvan.

Mime
View raw message