couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Cottlehuber <d...@muse.net.nz>
Subject Re: [jira] [Resolved] (COUCHDB-1334) Indexer speedup (for non-native view servers)
Date Tue, 22 Nov 2011 23:20:23 GMT
On 16 November 2011 13:02, Filipe Manana (Resolved) (JIRA)
<jira@apache.org> wrote:
>
>     [ https://issues.apache.org/jira/browse/COUCHDB-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
>
> Filipe Manana resolved COUCHDB-1334.
> ------------------------------------
>
>       Resolution: Fixed
>    Fix Version/s:     (was: 1.2)
>                   1.3
>
> Latest patch applied against master.
>
>> Indexer speedup (for non-native view servers)
>> ---------------------------------------------
>>
>>                 Key: COUCHDB-1334
>>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1334
[snip]
>> The following 2 patches significantly improve view index generation/update time and
reduce CPU consumption.
>> The first patch makes the view updater's batching more efficient, by ensuring each
btree bulk insertion adds/removes a minimum of N (=100) key/value pairts. This also makes
the index file size grow not so fast with old data (old btree nodes basically). This behaviour
is already done in master/trunk in the new indexer (by Paul Davis).
>> The second patch maximizes the throughput with an external view server (such as couchjs).
Basically it makes the pipe (erlang port) communication between the Erlang VM (couch_os_process
basically) and the view server more efficient since the 2 sides spend less time block on reading
from the pipe.

Hi Filipe,

Just a heads up, but I am consistently having the Erlang VM hang when
doing a master build today. Clearly, my issue may not be related to
this patch but I can't see anything else after a quick look in git
history that stands out.

As this uses the same Spidermonkey, Erlang R14B03 + patches etc as for
all my other builds, I am assuming this is a bug triggered by CouchDB
in the VM somewhere.

Hopefully tomorrow I'll both isolate the commit where things go awry,
and also find a way of getting some logging or a test case -- it would
be great to have a fix for this go into OTP/R15.

If you have time, or some advice to share, the binary build is at
https://www.dropbox.com/s/jeifcxpbtpo78ak/Snapshots/20111122

Thanks
Dave

Mime
View raw message