incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: couchdb on 8 core redhat/fedora
Date Mon, 10 Aug 2009 15:55:55 GMT
On Mon, Aug 10, 2009 at 11:46 AM, Norman Barker<norman.barker@gmail.com> wrote:
> Paul,
>
> a post-write filter is to change the response from a call to update,
> where as a _show is for generating a particular response.  I was
> wondering about being able to add interceptor handlers (filters in the
> servlet world).
>

Oh. As part of the _update work there was a feature request to return
an arbitrary response. Not sure if that'll make the first
implementation or not.

> The pmap example was nice, but as you have commented writes are
> serialized so I will start again! I will look at seeing if I can
> parallize the json parsing.
>

If you're building from source, try something like:

./bootstrap && ERLC_FLAGS=+native ./configure && make dev && ./utils/run

That should give you a nice JSON performance boost assuming your
Erlang is HiPE enabled. It'll just complain and build without if its
not.

HTH,
Paul Davis

> Norman
>
> On Mon, Aug 10, 2009 at 8:00 AM, Paul Davis<paul.joseph.davis@gmail.com> wrote:
>> On Mon, Aug 10, 2009 at 9:53 AM, Norman Barker<norman.barker@gmail.com> wrote:
>>> I was a little surprised by this, and did a quick grep on spawn on the
>>> couchdb code base, it isn't used that much and I wonder why. In
>>> particular for bulk doc updates I wonder why pmap isn't used (perhaps
>>> I am missing something), for my project I have needed to write a
>>> custom updater, a pre-writer, (great modular design now btw in 0.9 I
>>> was able to hook it all up - updater, query server from the ini file)
>>> and I am going up to a javascript function to parse an incoming
>>> document (not JSON) into multiple JSON documents I am then using a
>>> pmap to write this to the db, doesn't seem to be that complicated and
>>> I should then get an increase in speed to compensate for the initial
>>> javascript parsing delay.
>>>
>>> Could all (or most) use of the plain map be replaced with a pmap in
>>> couchdb or am I missing something?
>>>
>>> Not to hijack this thread, but I have written a pre-writer, are there
>>> any plans to add filters (pre and post) on the couchdb roadmap?
>>>
>>> As a J2EE developer, I still see the benefit of the Erlang VM - easier
>>> to develop 'fast' applications in for sure.
>>>
>>> thanks,
>>>
>>> Norman
>>>
>>
>> Couple points,
>>
>> All writes to disk are serialized, so using a pmap to save docs isn't
>> going to save you much unless you're parallelizing the JSON parsing.
>>
>> I'm not sure what you mean by pre and post filters. I wrote a first
>> pass at being able to mutate documents when saving last week or so.
>> Post filters sounds alot like _show.
>>
>> HTH,
>> Paul Davis
>>
>

Mime
View raw message