couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Continous Replication hangs with "start"
Date Wed, 04 Nov 2009 13:42:31 GMT
Hi Simon, looks like replication is failing on the following URL:

http://10.1.1.31:5984/gangstercluster_1/ping_http:%2F%2Fredemption.intranet.struktur.de:2425?open_revs=[\

"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true

It gets back a response that looks like

HTTP/1.1 500 Internal Server Error
Server: CouchDB/0.10.0 (Erlang OTP/R13B)
Date: Wed, 04 Nov 2009 11:10:19 GMT
Content-Type: application/json
Content-Length: 114
Cache-Control: must-revalidate

{\"error\":\"json_encode\",\"reason\":\"{bad_term, 
{26621,<<47,146,160,252,77,28,71,180,40,233,173,84,97,192,173,47>>}}\"} 
\n"

You could try the URL for yourself and confirm this to be the case.   
The problem is probably that revision 26621-.... of this document is  
missing.  0.10.0 had a bug where if you explicitly requested a missing  
revision, you'd get a 500 Internal Server Response.  The replicator  
isn't in the habit of making these kinds of requests, but it's  
probably some interaction with the full disk problem.

I fixed this bug in r829919 (trunk) or r829924 (0.10.x branch, didn't  
make 0.10.0), so if you upgraded it would likely allow replication to  
proceed.  Otherwise, I think your recourse is to make some update to  
this document so that the replicator doesn't try to request this  
revision anymore.  Hope it helps,

Adam

On Nov 4, 2009, at 6:15 AM, Simon Eisenmann wrote:

> Hi,
>
> i have been running continuous replication for the past three weeks on
> three nodes each replicating to each other with continuous changes on
> every node. All running fine until one node crashed cause of a full
> disk.
>
> I emptied disk space and restarted this node. This replication never
> seems to start again. Status in futon keep saying "Starting".
>
> I tried to restart all the nodes with no luck, all having the same
> issues.
>
> See the errors in the couchdb.log below.
>
> I am running CouchDB 0.10.0 on Erlang R13B.
>
> I hope somebody has some sort of idea what is going on.
>
>
> Thanks
> Simon
>
>
> [Wed, 04 Nov 2009 11:10:19 GMT] [error] [<0.2114.0>] ** Generic server
> <0.2114.0> terminating
> ** Last message in was {tcp,#Port<0.2219>,
>                            <<"HTTP/1.1 500 Internal Server Error\r
> \nServer: CouchDB/0.10.0 (Erlang OTP/R13B)\r\nDate: Wed, 04 Nov 2009
> 11:10:19 GMT\r\nContent-Type: application/json\r\nContent-Length:  
> 114\r
> \nCache-Control: must-revalidate\r\n\r\n{\"error\":\"json_encode\",
> \"reason\":
> \"{bad_term, 
> {26621,<<47,146,160,252,77,28,71,180,40,233,173,84,97,192,173,47>>}} 
> \"}\n">>}
> ** When Server state ==
> {state,"10.1.1.31",5984,false,undefined,[],false,
>                            #Port<0.2219>,
>                            {[{request,
>                                  {url,
>
> "http://10.1.1.31:5984/gangstercluster_1/ping_http:%2F%
> 2Fredemption.intranet.struktur.de:2425?open_revs= 
> [\"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true",
>
> "10.1.1.31",5984,undefined,undefined,
>                                      "/gangstercluster_1/ping_http: 
> %2F%
> 2Fredemption.intranet.struktur.de:2425?open_revs= 
> [\"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true",
>                                      http},
>                                  get,
>                                  [{response_format,binary},
>                                   {inactivity_timeout,30000}],
>                                  {<0.2084.0>,#Ref<0.0.3.30137>},
>                                  undefined,false,
>                                  {1257,333019,329755},
>
> 1048576,false,undefined,undefined,binary}],
>                             []},
>                            {request,
>                                {url,
>
> "http://10.1.1.31:5984/gangstercluster_1/ping_http:%2F%
> 2Fredemption.intranet.struktur.de:2425?open_revs= 
> [\"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true",
>
> "10.1.1.31",5984,undefined,undefined,
>                                    "/gangstercluster_1/ping_http:%2F%
> 2Fredemption.intranet.struktur.de:2425?open_revs= 
> [\"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true",
>                                    http},
>                                get,
>                                [{response_format,binary},
>                                 {inactivity_timeout,30000}],
>                                {<0.2084.0>,#Ref<0.0.3.30137>},
>                                undefined,false,
>                                {1257,333019,329755},
>
> 1048576,false,undefined,undefined,binary},
>                            get_body,"200",<<"[">>,0,0,
>                            [{"Transfer-Encoding","chunked"},
>                             {"Server","CouchDB/0.10.0 (Erlang
> OTP/R13B)"},
>                             {"Date","Wed, 04 Nov 2009 11:10:19 GMT"},
>                             {"Content-Type","application/json"},
>                             {"Cache-Control","must-revalidate"}],
>                            false,undefined,undefined,true,chunked,
>                            chunk_start,<<>>,0,262182,1,undefined}
> ** Reason for termination ==
> ** {function_clause,[{ibrowse_http_client,to_ascii,"r"},
>                     {ibrowse_http_client,hexlist_to_integer,3},
>                     {ibrowse_http_client,parse_11_response,2},
>                     {ibrowse_http_client,handle_sock_data,2},
>                     {gen_server,handle_msg,5},
>                     {proc_lib,init_p_do_apply,3}]}
>
>
> [Wed, 04 Nov 2009 11:10:19 GMT] [error] [<0.2114.0>]
> {error_report,<0.24.0>,
>    {<0.2114.0>,crash_report,
>     [[{initial_call,{ibrowse_http_client,init,['Argument__1']}},
>       {pid,<0.2114.0>},
>       {registered_name,[]},
>       {error_info,
>           {exit,
>               {function_clause,
>                   [{ibrowse_http_client,to_ascii,"r"},
>                    {ibrowse_http_client,hexlist_to_integer,3},
>                    {ibrowse_http_client,parse_11_response,2},
>                    {ibrowse_http_client,handle_sock_data,2},
>                    {gen_server,handle_msg,5},
>                    {proc_lib,init_p_do_apply,3}]},
>
> [{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]}},
>       {ancestors,[<0.139.0>,ibrowse,<0.1.0>]},
>       {messages,[]},
>       {links,[<0.139.0>]},
>       {dictionary,
>           [{my_trace_flag,false},
>            {ibrowse_trace_token,["10.1.1.31",58,"5984"]},
>            {http_prot_vsn,"HTTP/1.1"},
>            {conn_close,"false"}]},
>       {trap_exit,false},
>       {status,running},
>       {heap_size,4181},
>       {stack_size,24},
>       {reductions,1570}],
>      []]}}
>
>
> [Wed, 04 Nov 2009 11:12:25 GMT] [error] [<0.2339.0>] ** Generic server
> <0.2339.0> terminating
> ** Last message in was {tcp,#Port<0.2310>,
>                            <<"HTTP/1.1 500 Internal Server Error\r
> \nServer: CouchDB/0.10.0 (Erlang OTP/R13B)\r\nDate: Wed, 04 Nov 2009
> 11:12:25 GMT\r\nContent-Type: application/json\r\nContent-Length:  
> 114\r
> \nCache-Control: must-revalidate\r\n\r\n{\"error\":\"json_encode\",
> \"reason\":
> \"{bad_term, 
> {26621,<<47,146,160,252,77,28,71,180,40,233,173,84,97,192,173,47>>}} 
> \"}\n">>}
> ** When Server state ==
> {state,"10.1.1.31",5984,false,undefined,[],false,
>                            #Port<0.2310>,
>                            {[{request,
>                                  {url,
>
> "http://10.1.1.31:5984/gangstercluster_1/ping_http:%2F%
> 2Fredemption.intranet.struktur.de:2425?open_revs= 
> [\"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true",
>
> "10.1.1.31",5984,undefined,undefined,
>                                      "/gangstercluster_1/ping_http: 
> %2F%
> 2Fredemption.intranet.struktur.de:2425?open_revs= 
> [\"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true",
>                                      http},
>                                  get,
>                                  [{response_format,binary},
>                                   {inactivity_timeout,30000}],
>                                  {<0.2084.0>,#Ref<0.0.3.120029>},
>                                  undefined,false,
>                                  {1257,333145,499104},
>
> 1048576,false,undefined,undefined,binary}],
>                             []},
>                            {request,
>                                {url,
>
> "http://10.1.1.31:5984/gangstercluster_1/ping_http:%2F%
> 2Fredemption.intranet.struktur.de:2425?open_revs= 
> [\"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true",
>
> "10.1.1.31",5984,undefined,undefined,
>                                    "/gangstercluster_1/ping_http:%2F%
> 2Fredemption.intranet.struktur.de:2425?open_revs= 
> [\"26621-2f92a0fc4d1c47b428e9ad5461c0ad2f\"]&revs=true&latest=true",
>                                    http},
>                                get,
>                                [{response_format,binary},
>                                 {inactivity_timeout,30000}],
>                                {<0.2084.0>,#Ref<0.0.3.120029>},
>                                undefined,false,
>                                {1257,333145,499104},
>
> 1048576,false,undefined,undefined,binary},
>                            get_body,"200",<<"[">>,0,0,
>                            [{"Transfer-Encoding","chunked"},
>                             {"Server","CouchDB/0.10.0 (Erlang
> OTP/R13B)"},
>                             {"Date","Wed, 04 Nov 2009 11:12:25 GMT"},
>                             {"Content-Type","application/json"},
>                             {"Cache-Control","must-revalidate"}],
>                            false,undefined,undefined,true,chunked,
>                            chunk_start,<<>>,0,311334,1,undefined}
> ** Reason for termination ==
> ** {function_clause,[{ibrowse_http_client,to_ascii,"r"},
>                     {ibrowse_http_client,hexlist_to_integer,3},
>                     {ibrowse_http_client,parse_11_response,2},
>                     {ibrowse_http_client,handle_sock_data,2},
>                     {gen_server,handle_msg,5},
>                     {proc_lib,init_p_do_apply,3}]}
>
>
>
> -- 
> Simon Eisenmann
>
> [ mailto:simon@struktur.de ]
>
> [ struktur AG | Kronenstra├če 22a | D-70173 Stuttgart ]
> [ T. +49.711.896656.0 | F.+49.711.89665610 ]
> [ http://www.struktur.de | mailto:info@struktur.de ]


Mime
View raw message