Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4FA7EDB85 for ; Mon, 6 Aug 2012 11:24:15 +0000 (UTC) Received: (qmail 32467 invoked by uid 500); 6 Aug 2012 11:24:13 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 32229 invoked by uid 500); 6 Aug 2012 11:24:08 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 32206 invoked by uid 99); 6 Aug 2012 11:24:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Aug 2012 11:24:08 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FSL_RCVD_USER,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of m.bykov@gmail.com designates 209.85.213.180 as permitted sender) Received: from [209.85.213.180] (HELO mail-yx0-f180.google.com) (209.85.213.180) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Aug 2012 11:24:03 +0000 Received: by yenq6 with SMTP id q6so2338305yen.11 for ; Mon, 06 Aug 2012 04:23:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=8G8jb+8sfSIEznvqIPC/VI2WkKqYiKklutuPIeREcFM=; b=CpbuEqujh8FCimSbW/e6O9RLbbbS1KiFxJbHW/KWVoYPvxIfafybfOHngdn5R25y/h IjkEumjaaOMSg7EXrqLgOcr/xgC8a9RZjQWHAd7+ajVHGn9Dn9dPO+7pnhgF1ZuhHtUF ZRYzpZBNqv8v/j+4cI9pxlHQUbQ8ByMniGDslrxoMM0welyMXhGg/6Sff82hmhYlYKux XDLbE9YiIRDmrK7M9h1Wh8kiTeHnGsKP+xyzcQ0EeTOtswfh+dlggqR2eTgEkAiIABlT OmyJM/gyP3Ug/yWHipEilMQhBCb0NfeRRa48nSq+8wBLvENwH4I3XwrRIhxr+30mgVjf 8nvw== MIME-Version: 1.0 Received: by 10.50.85.228 with SMTP id k4mr5055451igz.22.1344252222203; Mon, 06 Aug 2012 04:23:42 -0700 (PDT) Received: by 10.64.39.230 with HTTP; Mon, 6 Aug 2012 04:23:41 -0700 (PDT) In-Reply-To: References: Date: Mon, 6 Aug 2012 15:23:41 +0400 Message-ID: Subject: Re: I replicate _replicator by chance From: Michael Bykov To: user@couchdb.apache.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org 2012/8/6 Benoit Chesneau : > On Mon, Aug 6, 2012 at 11:55 AM, Michael Bykov wrote: >> 2012/8/6 Benoit Chesneau : >>> On Mon, Aug 6, 2012 at 11:35 AM, Michael Bykov wrot= e: >>>> 2012/8/6 Benoit Chesneau : >>>>> did you replicate the _replicator db as an admin ? >>>> >>>> Hi Benoit, >>>> >>>> Yes. >>>> >>>> M. >>>> >>> >>> I mean in the replicator document which auth are you giving? Can you >>> paste your replicator doc here? >>> >>> - beno=C3=AEt >> >> Here it is - >> > > This is not the replication task document :) >> >> function(newDoc, oldDoc, userCtx) { >> function reportError(error_msg) { >> log('Error writing document `' + newDoc._id + >> '\' to the replicator database: ' + error_msg); >> throw({forbidden: error_msg}); >> } >> >> function validateEndpoint(endpoint, fieldName) { >> if ((typeof endpoint !=3D=3D 'string') && >> ((typeof endpoint !=3D=3D 'object') || (endpoint =3D=3D= =3D null))) { >> >> reportError('The `' + fieldName + '\' property must exis= t' + >> ' and be either a string or an object.'); >> } >> >> if (typeof endpoint =3D=3D=3D 'object') { >> if ((typeof endpoint.url !=3D=3D 'string') || !endpoint.= url) { >> reportError('The url property must exist in the `' + >> fieldName + '\' field and must be a non-empty st= ring.'); >> } >> >> if ((typeof endpoint.auth !=3D=3D 'undefined') && >> ((typeof endpoint.auth !=3D=3D 'object') || >> endpoint.auth =3D=3D=3D null)) { >> >> reportError('`' + fieldName + >> '.auth\' must be a non-null object.'); >> } >> >> if ((typeof endpoint.headers !=3D=3D 'undefined') && >> ((typeof endpoint.headers !=3D=3D 'object') || >> endpoint.headers =3D=3D=3D null)) { >> >> reportError('`' + fieldName + >> '.headers\' must be a non-null object.'); >> } >> } >> } >> >> var isReplicator =3D (userCtx.roles.indexOf('_replicator') >=3D = 0); >> var isAdmin =3D (userCtx.roles.indexOf('_admin') >=3D 0); >> >> if (oldDoc && !newDoc._deleted && !isReplicator && >> (oldDoc._replication_state =3D=3D=3D 'triggered')) { >> reportError('Only the replicator can edit replication docume= nts ' + >> 'that are in the triggered state.'); >> } >> >> if (!newDoc._deleted) { >> validateEndpoint(newDoc.source, 'source'); >> validateEndpoint(newDoc.target, 'target'); >> >> if ((typeof newDoc.create_target !=3D=3D 'undefined') && >> (typeof newDoc.create_target !=3D=3D 'boolean')) { >> >> reportError('The `create_target\' field must be a boolea= n.'); >> } >> >> if ((typeof newDoc.continuous !=3D=3D 'undefined') && >> (typeof newDoc.continuous !=3D=3D 'boolean')) { >> >> reportError('The `continuous\' field must be a boolean.'= ); >> } >> >> if ((typeof newDoc.doc_ids !=3D=3D 'undefined') && >> !isArray(newDoc.doc_ids)) { >> >> reportError('The `doc_ids\' field must be an array of >> strings.'); >> } >> >> if ((typeof newDoc.filter !=3D=3D 'undefined') && >> ((typeof newDoc.filter !=3D=3D 'string') || !newDoc.filt= er)) { >> >> reportError('The `filter\' field must be a non-empty str= ing.'); >> } >> >> if ((typeof newDoc.query_params !=3D=3D 'undefined') && >> ((typeof newDoc.query_params !=3D=3D 'object') || >> newDoc.query_params =3D=3D=3D null)) { >> >> reportError('The `query_params\' field must be an object= .'); >> } >> >> if (newDoc.user_ctx) { >> var user_ctx =3D newDoc.user_ctx; >> >> if ((typeof user_ctx !=3D=3D 'object') || (user_ctx =3D= =3D=3D null)) { >> reportError('The `user_ctx\' property must be a ' + >> 'non-null object.'); >> } >> >> if (!(user_ctx.name =3D=3D=3D null || >> (typeof user_ctx.name =3D=3D=3D 'undefined') || >> ((typeof user_ctx.name =3D=3D=3D 'string') && >> user_ctx.name.length > 0))) { >> >> reportError('The `user_ctx.name\' property must be a= ' + >> 'non-empty string or null.'); >> } >> >> if (!isAdmin && (user_ctx.name !=3D=3D userCtx.name)) { >> reportError('The given `user_ctx.name\' is not valid= '); >> } >> >> if (user_ctx.roles && !isArray(user_ctx.roles)) { >> reportError('The `user_ctx.roles\' property must be = ' + >> 'an array of strings.'); >> } >> >> if (!isAdmin && user_ctx.roles) { >> for (var i =3D 0; i < user_ctx.roles.length; i++) { >> var role =3D user_ctx.roles[i]; >> >> if (typeof role !=3D=3D 'string' || role.length = =3D=3D=3D 0) { >> reportError('Roles must be non-empty strings= .'); >> } >> if (userCtx.roles.indexOf(role) =3D=3D=3D -1) { >> reportError('Invalid role (`' + role + >> '\') in the `user_ctx\''); >> } >> } >> } >> } else { >> if (!isAdmin) { >> reportError('The `user_ctx\' property is missing (it= is ' + >> 'optional for admins only).'); >> } >> } >> } else { >> if (!isAdmin) { >> if (!oldDoc.user_ctx || (oldDoc.user_ctx.name !=3D=3D >> userCtx.name)) { >> reportError('Replication documents can only be >> deleted by ' + >> 'admins or by the users who created them.'); >> } >> } >> } >> } >> >> >> >> >> >> >> >> >> >>>> >>>> >>>>> >>>>> On Mon, Aug 6, 2012 at 1:46 AM, Michael Bykov wro= te: >>>>>> 2012/8/5 Dave Cottlehuber : >>>>>>> On 5 August 2012 19:47, Michael Bykov wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I by chance replicate "_replicator" to a destination database. Mor= e >>>>>>>> exactly, I click "replicate" button, but nothing happens. >>>>>>>> >>>>>>>> But now, when I replicate my local db, it wrote in a log on each d= ocument: >>>>>>>> >>>>>>>> [error] [<0.28457.5>] Replicator: couldn't write document >>>>>>>> `4fc8269be10198e2bfa2a137fe1a9333`, revision >>>>>>>> `1-4a8baba0dd74f92ec0da91e8fff84774`, to target database >>>>>>>> `http://admin:*****@localhost:5986/diglossa/`. Error: `forbidden`, >>>>>>>> reason: `The `source' property must exist and be either a string o= r an >>>>>>>> object.` >>>>>>>> >>>>>>>> And in Futon: >>>>>>>> >>>>>>>> {"session_id":"d7c097b8d822f707725b1b79b5bc3bc8","start_time":"Sun= , 05 >>>>>>>> Aug 2012 17:34:34 GMT","end_time":"Sun, 05 Aug 2012 17:34:42 >>>>>>>> GMT","start_last_seq":262336,"end_last_seq":262672,"recorded_seq":= 262672,"missing_checked":280,"missing_found":280,"docs_read":280,"docs_writ= ten":0,"doc_write_failures":280} >>>>>>>> >>>>>>>> I compact and cleanup both databases, it did not help. >>>>>>>> >>>>>>>> Please, suggest, what can be done? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> =D0=9C. >>>>>>>> >>>>>>>> http://diglossa.ru >>>>>>>> xmpp://m.bykov@jabber.ru >>>>>>> >>>>>>> Hi Michael, >>>>>>> >>>>>>> The thing stopping writing your docs is a validation function (clue >>>>>>> "`The `source' property must exist and be either a string or an >>>>>>> object" is not in the couch source tree). I'm guessing a little but= I >>>>>>> think your _replicator db at one end now has "normal" design docs i= n >>>>>>> it. >>>>>>> >>>>>>> What I'd do now is: >>>>>>> >>>>>>> 1. rename the _replicator.couch db file at the problem end to somet= hing else. >>>>>>> 2. restart couchdb (get a new, clean _replicator) >>>>>>> 3. if you need to, log into futon as admin and delete all ddocs in = the >>>>>>> renamed, foobared _replicator and rename it back. You'll likely onl= y >>>>>>> need this if you have continuous replications, or ones that should = be >>>>>>> present on a restart of couchdb. >>>>>>> 4. any more cleanup as reqd by your application. >>>>>>> >>>>>>> I think that's enough to get you started. >>>>>>> >>>>>>> A+ >>>>>>> Dave >>>>>> >>>>>> >>>>>> Hi Dave, thank you! >>>>>> >>>>>> yes, _replicator.couch files had different sizes, 4.1 kb on local en= d >>>>>> and 8.2 on remote. >>>>>> >>>>>> I removed both _replicator.couch and restart both couchdb. but alas, >>>>>> it did not help. >>>>>> >>>>>> in logs on my side (each document) >>>>>> >>>>>> [error] [<0.3848.0>] Replicator: couldn't write document xxx to targ= et database >>>>>> >>>>>> but on remote side >>>>>> >>>>>> [info] [<0.253.0>] OS Process #Port<0.3042> Log :: Error writing >>>>>> document `1a1f377aaa9a5f0125b2344c20087eb7' to the replicator >>>>>> database: The `source' property must exist and be either a string or >>>>>> an object. >>>>>> >>>>>> Why it write 'replicator database'? >>>>>> >>>>>> By the way, remote end had a lot of strange for this place files and >>>>>> directories, for example .bashrc, .ssh, .bash_logout. .emacs.d/ >>>>>> >>>>>> I have deleted all of them, and restart it again. Now it has the sam= e >>>>>> files as my local end. >>>>>> >>>>>> Permissions are correct, as far as I can see. >>>>>> >>>>>> I have deleted my new docs from local db, replication goes ok, in Fu= ton >>>>>> >>>>>> "missing_checked":280,"missing_found":280,"docs_read":280,"docs_writ= ten":280,"doc_write_failures":0} >>>>>> >>>>>> and then I have created this docs again and had again >>>>>> >>>>>> "missing_checked":280,"missing_found":280,"docs_read":280,"docs_writ= ten":0,"doc_write_failures":280} >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> =D0=9C. >>>>>> >>>>>> http://diglossa.ru >>>>>> xmpp://m.bykov@jabber.ru >>>> >>>> >>>> >>>> -- >>>> =D0=9C. >>>> >>>> http://diglossa.ru >>>> xmpp://m.bykov@jabber.ru >> >> >> >> -- >> =D0=9C. >> >> http://diglossa.ru >> xmpp://m.bykov@jabber.ru Yes, it is validate_doc_updat function. So I am afraid I have no replication task document at all. What I have in _replicator (created from scratch just now) is only: _id _design/_replicator _rev 1-5bfa2c99eefe2b2eb4962db50aa3cfd4 language javascript validate_doc_update function(newDoc, oldDoc, userCtx) { function reportError(error_msg) { log('Error writing document `' + newDoc._id + ... And absolutly the same I have on remote end. I did not create some persistent replication tasks by hand, I just remove _replicator.couch and restart couchdb. Sorry, it is obviously I am missing something important, but what? --=20 =D0=9C. http://diglossa.ru xmpp://m.bykov@jabber.ru