incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Katz <dam...@apache.org>
Subject Re: Largest CouchDB dbs?
Date Mon, 03 Nov 2008 10:50:44 GMT

On Nov 3, 2008, at 4:34 AM, Jan Lehnardt wrote:

>
> On Nov 3, 2008, at 10:26, Ask Bjørn Hansen wrote:
>
>>
>> On Nov 2, 2008, at 21:19, Paul Davis wrote:
>>
>>> View updates are admittedly slower than we'd like. There's planned
>>> work on making this sort of thing parallelized to automagically fill
>>> out multi-node clusters. Last I remember this is a 'probably 1.0'
>>> feature though.
>>
>>
>> I wonder if some of the slowness is from running it through JS?  
>> Perl has an extremely fast JSON parser (JSON::XS), would there be  
>> any potential optimization in using an alternate view language like  
>> that - or is most of the time spent elsewhere?
>
> Another part is the Erlang to JSON conversion that is known to be  
> slow-ish. I don't know
> which side of the conversion weighs more though.

I have to assume our Erlang json parser is much slower than the  
spidermonkey one, which is just a call to eval(). Based on what I've  
seen when profiling Erlang code, a built-in parser will give us a big  
speed-up in view indexing.

-Damien


> We are working with the Erlang
> community and developers to get a C-based JSON encoder into Erlang.  
> Since building
> a view server is relatively straightforward, giving a Perl version a  
> shot is probably worth it.
>
> Which Erlang version are you running. R11B-x is known for causing  
> view-build slowness.
> R12B-3 and R12B-4 are good.
>
>
>> I don't expect more than a few tens of thousands of new documents a  
>> day though, so other than on index changes this isn't such a big  
>> deal.
>
> That sounds about correct.
>
> Cheers
> Jan
> --
>


Mime
View raw message