incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <robert.new...@gmail.com>
Subject Re: external handlers are not in sync with commits
Date Tue, 27 Jul 2010 21:57:04 GMT
Ah, I've confused external with update_notification. You can contact
couchdb-lucene on its own port which handles concurrent calls but,
yes, calls via an external are serialized (which sucks).

B.


On Tue, Jul 27, 2010 at 10:48 PM, Nils Breunese <N.Breunese@vpro.nl> wrote:
> We ran into performance problems when we put a site that was pretty couchdb-lucene heavy
into production. The query times in couchdb-lucene were fast, but we believe it was all those
concurrent queries that killed performance, since they all had to go through this single externals
pipe to couchdb-lucene. Using the _changes feed as couchdb-lucene's input sounds like a good
idea, but then couchdb-lucene queries also need to be going directly to the couchdb-lucene
instance, right? I believe it supports that now, but applications may need to be modified
for this. Or is there a way that this could still go through the current _fti URL's (without
adding some mod_rewrite like magic).
>
> Nils.
> ________________________________________
> Van: Robert Newson [robert.newson@gmail.com]
> Verzonden: dinsdag 27 juli 2010 23:41
> Aan: user@couchdb.apache.org
> Onderwerp: Re: external handlers are not in sync with commits
>
> reading _changes instead of using the (deprecated?) externals feature
> would avoid the problem?
>
> B.
>
> On Tue, Jul 27, 2010 at 10:37 PM, J Chris Anderson <jchris@apache.org> wrote:
>>
>> On Jul 27, 2010, at 2:30 PM, Norman Barker wrote:
>>
>>> Hi,
>>>
>>> I have written couchdb-clucene
>>> (http://github.com/normanb/couchdb-clucene) and am doing a lot of
>>> testing with heavy datasets where I am sending a bulk doc request with
>>> 10 docs at a time, a couple of these every second for a couple of
>>> minutes.
>>>
>>> Very quickly couchdb backs up and hogs the cpu since the database
>>> commit and return doesn't wait for an external handler to do its job.
>>> The model of fire and forget is fine and I like it, very similar to
>>> JMS, however since the external process is a singleton it has to be
>>> very quick to keep up with load or the system slowly backs up.
>>>
>>> Is there a way to either define a pool of externals, or to change the
>>> default behaviour from fire and forget?
>>>
>>
>> Yes, but involves feature development for CouchDB. Essentially we need the externals
protocol to be non-blocking. There is a thread on dev@ that touches on this. I'm not sure
who wants to own making the patch, but the technical requirements are pretty well known.
>>
>> Thank you for working on something so awesome!
>>
>> Chris
>>
>>> thanks,
>>>
>>> Norman
>>
>>
>
> De informatie vervat in deze  e-mail en meegezonden bijlagen is uitsluitend bedoeld
voor gebruik door de geadresseerde en kan vertrouwelijke informatie bevatten. Openbaarmaking,
vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is voorbehouden
aan geadresseerde. De VPRO staat niet in voor de juiste en volledige overbrenging van de inhoud
van een verzonden e-mail, noch voor tijdige ontvangst daarvan.
>

Mime
View raw message