incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <adam.kocolo...@gmail.com>
Subject Re: [jira] Updated: (COUCHDB-160) replication performance improvements
Date Fri, 14 Nov 2008 03:21:57 GMT
On Nov 11, 2008, at 9:54 PM, Adam Kocoloski wrote:

> I tried to do some testing with large (>1MB) binary attachments this  
> afternoon and ran into a lot of stability issues at high  
> concurrency.  Quite a few of the replicator's HTTP processes (both  
> GET and POST) died gruesome deaths with session_remotly_closed errors.

Hi, just to follow up, those session_remotly_closed errors were  
unrelated to the patch I posted.  The patch I posted definitely uses  
more memory for large documents than the trunk code since it buffers  
multiple docs at once, but I'd argue that trunk's handling of these  
large documents is suboptimal anyway.  I tried replicating a DB with  
10 2.3MB documents over HTTP; trunk required well over 100 MB to get  
this done.

Perhaps there's a way to stream attachments separately during  
replication instead of using attachments=true and going through the  
whole Base64 conversion.  Regards, Adam

Mime
View raw message