Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type;
  b=XidBpZncF9+pNvD9+kl8n/TGRGsKAF9l2HgKYMt1NJe9l/Dt8xJ8j9fuPqIiBaJStqPLe6XYdJD/SnIN3Z+efJRd29mnWsomMrFxH+QIdV+RMdEtQnQcXlBHyVV+kw60F89xaoW5cs4YBX2T7OdA7zBG3lp81f+5K6iGnts/8MI=;
Message-ID: <1338937956.70626.YahooMailNeo@web39401.mail.mud.yahoo.com>
Date: Tue, 5 Jun 2012 16:12:36 -0700 (PDT)
From: Andreas Kemkes <a5sk4s@yahoo.com>
Reply-To: Andreas Kemkes <a5sk4s@yahoo.com>
Subject: Filtered replication in 1.1.1 and beyond: questions and request for
 advise
To: "user@couchdb.apache.org" <user@couchdb.apache.org>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="1534414624-1158161634-1338937956=:70626"

--1534414624-1158161634-1338937956=:70626
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

I'm trying to split up a monolithic couchdb instance into individual instan=
ces using filtered replication (following http://wiki.apache.org/couchdb/Re=
plication).=0A=0A(1) Using 1.1.1 pull replication fails after a while with =
(I suspect due to COUCHDB-1289):=0A=0A=0A[Tue, 05 Jun 2012 22:20:39 GMT] [e=
rror] [<0.31594.186>] changes loop timeout, no data received from http://lo=
calhost:5984/source/=0A[Tue, 05 Jun 2012 22:20:39 GMT] [error] [<0.31594.18=
6>] ** Generic server <0.31594.186> terminating=A0=0A** Last message in was=
 changes_timeout=0A** When Server state =3D=3D {state,nil,nil,=0A=A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [<0.31569.186>,=0A=A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{http_db,"http://localho=
st:5984/source/",[],=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0"_changes",=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0[{"User-Agent","CouchDB/1.1.1"},=0A=A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {"Accept","application/=
json"}],=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0[{"filter",<<"queries/target">>},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {"style",all_docs},=0A=A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {"heartbeat",10000},=0A=A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {"since",0}=
,=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {"f=
eed",normal}],=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0get,nil,=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0[{stream_to,{<0.31594.186>,once}},=0A=A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {response_format,binary}],=0A=
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A010,500,<=
0.31595.186>},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A00, =A0=A0=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0[=
{<<"source">>,=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0<<"http://localhost:5984/source">>},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {<<"target">>,=0A=A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0<<"http://...@localhost:5984/target"=
>>},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {<<"filt=
er">>,<<"queries/target">>}]],=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 39982,<0.31595.186>,=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 {1338,934681,255278},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 false,0,<<>>,=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 {<0.31598.186>,#Ref<0.0.191.224241>},=0A=A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {[],[]},=0A=A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 nil}=A0=0A** Reason for termination =
=3D=3D=A0=0A** changes_timeout=0AQuestion: =A0Is there a 1.1.1 version that=
 has this issue fixed?=0A=0AQuestion: =A0What patches would I need to apply=
 to make it work? There are 3 different patches linked in=A0COUCHDB-1289. =
=A0Do I need all 3?=0A=0A(2) Using 1.2 pull replication from the same 1.1.1=
 source via ssh tunneling fails as well:=0A=0A=0A[Mon, 04 Jun 2012 23:27:50=
 GMT] [info] [<0.7389.0>] recording a checkpoint for `http://localhost:5986=
/source/` -> `http://...:*****@localhost:5984/target/` at source update_seq=
 39982=0A[Mon, 04 Jun 2012 23:27:50 GMT] [info] [<0.7268.0>] 127.0.0.1 - - =
PUT /target/_local/9dcc3b03231843cea27bcd33ace50d51 201=0A[Mon, 04 Jun 2012=
 23:27:59 GMT] [error] [<0.7400.0>] Replicator, request GET to "http://loca=
lhost:5986/source/_changes?filter=3Dqueries%2Ftarget&feed=3Dnormal&style=3D=
all_docs&since=3D0&heartbeat=3D10000" failed due to error req_timedout=0A[M=
on, 04 Jun 2012 23:27:59 GMT] [info] [<0.7400.0>] Retrying _changes request=
 to source database http://localhost:5986/source/ with since=3D39982=0A=0A.=
.. (~ 10 retries)=0A=0A[Mon, 04 Jun 2012 23:35:09 GMT] [error] [<0.7389.0>]=
 ChangesReader process died with reason: {http_request_failed,=0A=A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 "GET",=0A=A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 "http://localhost:5986/source/_changes?filter=3Dqu=
eries%2Ftarget&feed=3Dnormal&style=3Dall_docs&since=3D39982&heartbeat=3D100=
00",=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {error,=0A=A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0req_timedout}}=0A[Mon, 04 Jun 2=
012 23:35:09 GMT] [error] [<0.7389.0>] Replication `9dcc3b03231843cea27bcd3=
3ace50d51` (`http://localhost:5986/source/` -> `http://...:*****@localhost:=
5984/target/`) failed: changes_reader_died=0A=0AQuestion: =A0Is that the sa=
me issue as above (source is still 1.1.1)?=0A=0AQuestion: =A0What other opt=
ions do I have to make it work?=0A(3) I would like to combine the filtered =
replication with continuous replication, but when I do:=0A=0Acurl -H 'Conte=
nt-Type: application/json' -X POST -d '{"source":"http://localhost:5986/sou=
rce","target":"http://...@localhost:5984/target","filter":"queries/target",=
"continuous":"true"}' http://localhost:5984/_replicate=0A=0A=0Athen I see t=
he following error:=0A=0A{"error":"case_clause","reason":"true"}=0Aand in t=
he logs:=0A=0A=0A[Tue, 05 Jun 2012 23:04:04 GMT] [error] [<0.258.0>] Uncaug=
ht error in HTTP request: {error,=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{case_c=
lause,<<"true">>}}=0A[Tue, 05 Jun 2012 23:04:04 GMT] [info] [<0.258.0>] Sta=
cktrace: [{couch_replicator_utils,=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 '-maybe_append_options/2-fun-0-',3,=0A=
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [{f=
ile,"couch_replicator_utils.erl"},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{line,143}]},=0A=A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {lists,foldl,3,=0A=A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [{file,"lists.e=
rl"},{line,1197}]},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 {couch_replicator_utils,replication_id,1,=0A=A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [{file,"couch_replicat=
or_utils.erl"},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0{line,55}]},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 {couch_replicator_utils,parse_rep_doc,2,=0A=A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [{file,"cou=
ch_replicator_utils.erl"},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{line,49}]},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {couch_httpd_replicator,handle_req,1,=
=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
[{file,"couch_httpd_replicator.erl"},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{line,34}]},=0A=A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {couch_httpd,handle_request_int=
,5,=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 [{file,"couch_httpd.erl"},{line,317}]},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 {mochiweb_http,headers,5,=0A=A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [{file,"mochiw=
eb_http.erl"},{line,136}]},=0A=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 {proc_lib,init_p_do_apply,3,=0A=A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 [{file,"proc_lib.erl"},{lin=
e,227}]}]=0A[Tue, 05 Jun 2012 23:04:04 GMT] [error] [<0.258.0>] Uncaught se=
rver error: {case_clause,<<"true">>}=0A[Tue=0AQuestion: =A0Is this possible=
 now?=0A=0AQuestion: =A0Could this be possible in the future?=0A=0A=0AAny a=
dvise on how to approach this differently is very welcome as well. =A0Also,=
 please let me know if I can provide more information. =A0Thanks in advance=
.=0A=0A-- Andreas
--1534414624-1158161634-1338937956=:70626--