couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Hanging replication (was Re: Replication Status Messages)
Date Wed, 04 Nov 2009 01:47:09 GMT
Thanks Matt & Matt, we should put a big warning sticker on the  
replicator saying "don't push attachments" :(  Getting push  
replication to efficiently handle attachments is a blocker for the  
next release as far as I'm concerned.  Best,

Adam

On Nov 3, 2009, at 7:58 PM, Matt Aimonetti wrote:

> I also experienced the same problems with replication + attachments.
>
> - Matt
>
> On Tue, Nov 3, 2009 at 4:51 PM, Matt Goodall  
> <matt.goodall@gmail.com> wrote:
>
>> 2009/11/3 Adam Kocoloski <kocolosk@apache.org>:
>>> Yeah, that's too long, something seems to have gone south.  If  
>>> possible
>> I'd
>>> try restarting CouchDB and then the replication.  Best,
>>
>> I've also been experiencing similar problems with replication hanging
>> recently so I set about reproducing it tonight.
>>
>> In the end it was quite simple. I was fairly sure it was related to
>> attachments so I created a database with a bunch of documents each
>> containing a largish attachment. Then, I replicated that to an empty
>> database on the same couchdb server but over HTTP.
>>
>> See attached shell script. Run with an init arg first time to setup a
>> source database; run without the init arg on subsequent runs to avoid
>> creating the source database. I'm running on Ubuntu 9.10 and the
>> latest CouchDB 0.10.x branch.
>>
>> Replication starts ok but then hangs. The status page does not update
>> and I start seeing "Uncaught error in HTTP request" errors in the
>> couch logs.
>>
>> [error] [<0.1236.0>] Uncaught error in HTTP request: {exit,normal}
>> [info] [<0.1236.0>] Stacktrace: [{mochiweb_request,send,2},
>>            {mochiweb_request,respond,2},
>>            {couch_httpd,send_response,4},
>>            {couch_httpd_db,do_db_req,2},
>>            {couch_httpd,handle_request,5},
>>            {mochiweb_http,headers,5},
>>            {proc_lib,init_p_do_apply,3}]
>>
>> CPU is typically close to 100% anyway but after the first error the
>> erlang process's memory usage rises significantly.
>>
>> The only way to kill the hung replication process seems to be to
>> restart the server.
>>
>> Interestingly, continuous replication seems to have the same problem
>> but recovers from the failure and eventually completes.
>>
>> - Matt
>>
>>
>>>
>>> Adam
>>>
>>> On Nov 3, 2009, at 3:37 PM, Zachary Zolton wrote:
>>>
>>>> Coolio, so how long should I wait for that status message to change
>>>> locally before I start fretting? It's been the same for like a  
>>>> good 10
>>>> minutes, though I'm pushing to a remote couchy host on the
>>>> intarwebs....
>>>>
>>>> (FYI, my local database is 7.1MB compacted, has 756 docs and is  
>>>> at seq#
>>>> 1318)
>>>>
>>>>
>>>> On Tue, Nov 3, 2009 at 2:32 PM, Adam Kocoloski  
>>>> <kocolosk@apache.org>
>>>> wrote:
>>>>>
>>>>> On Nov 3, 2009, at 3:23 PM, Zachary Zolton wrote:
>>>>>
>>>>>> Howdy,
>>>>>>
>>>>>> I've been watching a push replication in Futon that's been  
>>>>>> crunching
>>>>>> for a while with the following message:
>>>>>> "W Processed source update #176"
>>>>>>
>>>>>> Does this mean that the remote server has process my local  
>>>>>> revisions
>>>>>> up to sequence number 176? Also, should I be able to see an  
>>>>>> active
>>>>>> task, or something on the replication page, for the remote  
>>>>>> server?
>>>>>>
>>>>>> Any explanations of what to look for would be greatly  
>>>>>> appreciated!
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Zach
>>>>>
>>>>> Hi Zach, that's correct, everything up to source update #176  
>>>>> should be
>>>>> available on the target server at that point.
>>>>>
>>>>> There's no easy way to generate a status update on the target  
>>>>> server,
>>>>> because to the target the replication just looks like a series of
>> normal
>>>>> HTTP requests.  Cheers,
>>>>>
>>>>> Adam
>>>>>
>>>
>>>
>>


Mime
View raw message