incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Kemkes <a5s...@yahoo.com>
Subject Filtered replication in 1.1.1 and beyond: questions and request for advise
Date Tue, 05 Jun 2012 23:12:36 GMT
I'm trying to split up a monolithic couchdb instance into individual instances using filtered
replication (following http://wiki.apache.org/couchdb/Replication).

(1) Using 1.1.1 pull replication fails after a while with (I suspect due to COUCHDB-1289):


[Tue, 05 Jun 2012 22:20:39 GMT] [error] [<0.31594.186>] changes loop timeout, no data
received from http://localhost:5984/source/
[Tue, 05 Jun 2012 22:20:39 GMT] [error] [<0.31594.186>] ** Generic server <0.31594.186>
terminating 
** Last message in was changes_timeout
** When Server state == {state,nil,nil,
                            [<0.31569.186>,
                             {http_db,"http://localhost:5984/source/",[],
                                 "_changes",
                                 [{"User-Agent","CouchDB/1.1.1"},
                                  {"Accept","application/json"}],
                                 [{"filter",<<"queries/target">>},
                                  {"style",all_docs},
                                  {"heartbeat",10000},
                                  {"since",0},
                                  {"feed",normal}],
                                 get,nil,
                                 [{stream_to,{<0.31594.186>,once}},
                                  {response_format,binary}],
                                 10,500,<0.31595.186>},
                             0,   
                             [{<<"source">>,
                               <<"http://localhost:5984/source">>},
                              {<<"target">>,
                               <<"http://...@localhost:5984/target">>},
                              {<<"filter">>,<<"queries/target">>}]],
                            39982,<0.31595.186>,
                            {1338,934681,255278},
                            false,0,<<>>,
                            {<0.31598.186>,#Ref<0.0.191.224241>},
                            {[],[]},
                            nil} 
** Reason for termination == 
** changes_timeout
Question:  Is there a 1.1.1 version that has this issue fixed?

Question:  What patches would I need to apply to make it work? There are 3 different patches
linked in COUCHDB-1289.  Do I need all 3?

(2) Using 1.2 pull replication from the same 1.1.1 source via ssh tunneling fails as well:


[Mon, 04 Jun 2012 23:27:50 GMT] [info] [<0.7389.0>] recording a checkpoint for `http://localhost:5986/source/`
-> `http://...:*****@localhost:5984/target/` at source update_seq 39982
[Mon, 04 Jun 2012 23:27:50 GMT] [info] [<0.7268.0>] 127.0.0.1 - - PUT /target/_local/9dcc3b03231843cea27bcd33ace50d51
201
[Mon, 04 Jun 2012 23:27:59 GMT] [error] [<0.7400.0>] Replicator, request GET to "http://localhost:5986/source/_changes?filter=queries%2Ftarget&feed=normal&style=all_docs&since=0&heartbeat=10000"
failed due to error req_timedout
[Mon, 04 Jun 2012 23:27:59 GMT] [info] [<0.7400.0>] Retrying _changes request to source
database http://localhost:5986/source/ with since=39982

... (~ 10 retries)

[Mon, 04 Jun 2012 23:35:09 GMT] [error] [<0.7389.0>] ChangesReader process died with
reason: {http_request_failed,
                                                           
  "GET",
                                                           
  "http://localhost:5986/source/_changes?filter=queries%2Ftarget&feed=normal&style=all_docs&since=39982&heartbeat=10000",
                                                           
  {error,
                                                           
   req_timedout}}
[Mon, 04 Jun 2012 23:35:09 GMT] [error] [<0.7389.0>] Replication `9dcc3b03231843cea27bcd33ace50d51`
(`http://localhost:5986/source/` -> `http://...:*****@localhost:5984/target/`) failed:
changes_reader_died

Question:  Is that the same issue as above (source is still 1.1.1)?

Question:  What other options do I have to make it work?
(3) I would like to combine the filtered replication with continuous replication, but when
I do:

curl -H 'Content-Type: application/json' -X POST -d '{"source":"http://localhost:5986/source","target":"http://...@localhost:5984/target","filter":"queries/target","continuous":"true"}'
http://localhost:5984/_replicate


then I see the following error:

{"error":"case_clause","reason":"true"}
and in the logs:


[Tue, 05 Jun 2012 23:04:04 GMT] [error] [<0.258.0>] Uncaught error in HTTP request:
{error,
                                                     {case_clause,<<"true">>}}
[Tue, 05 Jun 2012 23:04:04 GMT] [info] [<0.258.0>] Stacktrace: [{couch_replicator_utils,
                                    '-maybe_append_options/2-fun-0-',3,
                                    [{file,"couch_replicator_utils.erl"},
                                     {line,143}]},
                                {lists,foldl,3,
                                    [{file,"lists.erl"},{line,1197}]},
                                {couch_replicator_utils,replication_id,1,
                                    [{file,"couch_replicator_utils.erl"},
                                     {line,55}]},
                                {couch_replicator_utils,parse_rep_doc,2,
                                    [{file,"couch_replicator_utils.erl"},
                                     {line,49}]},
                                {couch_httpd_replicator,handle_req,1,
                                    [{file,"couch_httpd_replicator.erl"},
                                     {line,34}]},
                                {couch_httpd,handle_request_int,5,
                                    [{file,"couch_httpd.erl"},{line,317}]},
                                {mochiweb_http,headers,5,
                                    [{file,"mochiweb_http.erl"},{line,136}]},
                                {proc_lib,init_p_do_apply,3,
                                    [{file,"proc_lib.erl"},{line,227}]}]
[Tue, 05 Jun 2012 23:04:04 GMT] [error] [<0.258.0>] Uncaught server error: {case_clause,<<"true">>}
[Tue
Question:  Is this possible now?

Question:  Could this be possible in the future?


Any advise on how to approach this differently is very welcome as well.  Also, please let
me know if I can provide more information.  Thanks in advance.

-- Andreas
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message