incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Kemkes <a5s...@yahoo.com>
Subject Re: Repost: Filtered replication in 1.1.1 and beyond: questions and request for advise
Date Wed, 13 Jun 2012 16:32:55 GMT
Thank you Filipe. That helps so much.

-- Andreas


________________________________
 From: Filipe David Manana <fdmanana@apache.org>
To: user@couchdb.apache.org; Andreas Kemkes <a5sk4s@yahoo.com> 
Sent: Tuesday, June 12, 2012 6:02 PM
Subject: Re: Repost: Filtered replication in 1.1.1 and beyond: questions and request for advise
 
On Wed, Jun 13, 2012 at 1:52 AM, Andreas Kemkes <a5sk4s@yahoo.com> wrote:
> I'm still very much interested in an answer to any of my questions below.  Help is much
appreciated.
>
> -- Andreas
>
> I'm trying to split up a monolithic couchdb instance into individual instances using
filtered replication (following http://wiki.apache.org/couchdb/Replication).
>
> (1) Using 1.1.1 pull replication fails after a while with (I suspect due to COUCHDB-1289):
>
>
> [Tue, 05 Jun 2012 22:20:39 GMT] [error] [<0.31594.186>] changes loop timeout, no
data received from http://localhost:5984/source/
> [Tue, 05 Jun 2012 22:20:39 GMT] [error] [<0.31594.186>] ** Generic server <0.31594.186>
terminating
> ** Last message in was changes_timeout
> ** When Server state == {state,nil,nil,
>                             [<0.31569.186>,
>                              {http_db,"http://localhost:5984/source/",[],
>                                  "_changes",
>                                  [{"User-Agent","CouchDB/1.1.1"},
>                                   {"Accept","application/json"}],
>                                  [{"filter",<<"queries/target">>},
>                                   {"style",all_docs},
>                                   {"heartbeat",10000},
>                                   {"since",0},
>                                   {"feed",normal}],
>                                  get,nil,
>                                  [{stream_to,{<0.31594.186>,once}},
>                                   {response_format,binary}],
>                                  10,500,<0.31595.186>},
>                              0,
>                              [{<<"source">>,
>                                <<"http://localhost:5984/source">>},
>                               {<<"target">>,
>                                <<"http://...@localhost:5984/target">>},
>                               {<<"filter">>,<<"queries/target">>}]],
>                             39982,<0.31595.186>,
>                             {1338,934681,255278},
>                             false,0,<<>>,
>                             {<0.31598.186>,#Ref<0.0.191.224241>},
>                             {[],[]},
>                             nil}
> ** Reason for termination ==
> ** changes_timeout
> Question:  Is there a 1.1.1 version that has this issue fixed?
>
> Question:  What patches would I need to apply to make it work? There are 3 different
patches linked in COUCHDB-1289.  Do I need all 3?
>
> (2) Using 1.2 pull replication from the same 1.1.1 source via ssh tunneling fails as
well:
>
>
> [Mon, 04 Jun 2012 23:27:50 GMT] [info] [<0.7389.0>] recording a checkpoint for
`http://localhost:5986/source/` -> `http://...:*****@localhost:5984/target/` at source
update_seq 39982
> [Mon, 04 Jun 2012 23:27:50 GMT] [info] [<0.7268.0>] 127.0.0.1 - - PUT /target/_local/9dcc3b03231843cea27bcd33ace50d51
201
> [Mon, 04 Jun 2012 23:27:59 GMT] [error] [<0.7400.0>] Replicator, request GET to
"http://localhost:5986/source/_changes?filter=queries%2Ftarget&feed=normal&style=all_docs&since=0&heartbeat=10000"
failed due to error req_timedout
> [Mon, 04 Jun 2012 23:27:59 GMT] [info] [<0.7400.0>] Retrying _changes request to
source database http://localhost:5986/source/ with since=39982
>
> ... (~ 10 retries)
>
> [Mon, 04 Jun 2012 23:35:09 GMT] [error] [<0.7389.0>] ChangesReader process died
with reason: {http_request_failed,
>                                                          
    "GET",
>                                                          
    "http://localhost:5986/source/_changes?filter=queries%2Ftarget&feed=normal&style=all_docs&since=39982&heartbeat=10000",
>                                                          
    {error,
>                                                          
     req_timedout}}
> [Mon, 04 Jun 2012 23:35:09 GMT] [error] [<0.7389.0>] Replication `9dcc3b03231843cea27bcd33ace50d51`
(`http://localhost:5986/source/` -> `http://...:*****@localhost:5984/target/`) failed:
changes_reader_died
>
> Question:  Is that the same issue as above (source is still 1.1.1)?

The relevant fixes only have effect if the source server runs a
release with the fix (1.2.0).

>
> Question:  What other options do I have to make it work?
> (3) I would like to combine the filtered replication with continuous replication, but
when I do:
>
> curl -H 'Content-Type: application/json' -X POST -d '{"source":"http://localhost:5986/source","target":"http://...@localhost:5984/target","filter":"queries/target","continuous":"true"}' http://localhost:5984/_replicate
>
>
> then I see the following error:
>
> {"error":"case_clause","reason":"true"}
> and in the logs:

Try passing "continuous": true  (as a boolean and not as a string) in
the replication object/document.

>
>
> [Tue, 05 Jun 2012 23:04:04 GMT] [error] [<0.258.0>] Uncaught error in HTTP request:
{error,
>                                                      {case_clause,<<"true">>}}
> [Tue, 05 Jun 2012 23:04:04 GMT] [info] [<0.258.0>] Stacktrace: [{couch_replicator_utils,
>                                     '-maybe_append_options/2-fun-0-',3,
>                                     [{file,"couch_replicator_utils.erl"},
>                                      {line,143}]},
>                                 {lists,foldl,3,
>                                     [{file,"lists.erl"},{line,1197}]},
>                                 {couch_replicator_utils,replication_id,1,
>                                     [{file,"couch_replicator_utils.erl"},
>                                      {line,55}]},
>                                 {couch_replicator_utils,parse_rep_doc,2,
>                                     [{file,"couch_replicator_utils.erl"},
>                                      {line,49}]},
>                                 {couch_httpd_replicator,handle_req,1,
>                                     [{file,"couch_httpd_replicator.erl"},
>                                      {line,34}]},
>                                 {couch_httpd,handle_request_int,5,
>                                     [{file,"couch_httpd.erl"},{line,317}]},
>                                 {mochiweb_http,headers,5,
>                                     [{file,"mochiweb_http.erl"},{line,136}]},
>                                 {proc_lib,init_p_do_apply,3,
>                                     [{file,"proc_lib.erl"},{line,227}]}]
> [Tue, 05 Jun 2012 23:04:04 GMT] [error] [<0.258.0>] Uncaught server error: {case_clause,<<"true">>}
> [Tue
> Question:  Is this possible now?
>
> Question:  Could this be possible in the future?
>
>
> Any advise on how to approach this differently is very welcome as well.  Also, please
let me know if I can provide more information.  Thanks in advance.
>
> -- Andreas



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message