From dev-return-48883-archive-asf-public=cust-asf.ponee.io@couchdb.apache.org Fri Nov 1 09:36:11 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 55467180626 for ; Fri, 1 Nov 2019 10:36:11 +0100 (CET) Received: (qmail 27874 invoked by uid 500); 1 Nov 2019 09:36:10 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 27788 invoked by uid 99); 1 Nov 2019 09:36:09 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Nov 2019 09:36:09 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id DEC2BC223F for ; Fri, 1 Nov 2019 09:36:08 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.274 X-Spam-Level: X-Spam-Status: No, score=0.274 tagged_above=-999 required=6.31 tests=[RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.972, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id h-r8TxCR6qQc for ; Fri, 1 Nov 2019 09:36:04 +0000 (UTC) Received-SPF: Softfail (mailfrom) identity=mailfrom; client-ip=95.143.172.184; helo=monoceres.uberspace.de; envelope-from=jan@apache.org; receiver= Received: from monoceres.uberspace.de (monoceres.uberspace.de [95.143.172.184]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 2E2DD7DDD2 for ; Fri, 1 Nov 2019 09:26:33 +0000 (UTC) Received: (qmail 26166 invoked from network); 1 Nov 2019 09:26:26 -0000 Received: from localhost (HELO ?IPv6:2a02:8106:1d:4a00:995b:7c07:eacf:bf37?) (127.0.0.1) by monoceres.uberspace.de with SMTP; 1 Nov 2019 09:26:26 -0000 From: Jan Lehnardt Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: [VOTE] Couch replicator _scheduler/jobs and _scheduler/docs API change Date: Fri, 1 Nov 2019 10:26:24 +0100 References: To: dev@couchdb.apache.org In-Reply-To: Message-Id: <64893234-BC59-4CD2-9A06-9532FF95AA11@apache.org> X-Mailer: Apple Mail (2.3445.104.11) Solid, +1! Best Jan =E2=80=94 > On 31. Oct 2019, at 20:45, Nick Vatamaniuc = wrote: >=20 > Hi everyone, >=20 > For CouchDB 3.0 I am proposing to add all the replication status > details from _active_tasks like "docs_written", "docs_read", etc to > _scheduler/jobs and _scheduler/docs endpoints. >=20 > Couch replicator scheduler has been out for a few years already. It > added flexibility, performance and other features. One of the goals of > the scheduler was to provide a better set of APIs for monitoring > replication job progress. Those APIs are _scheduler/jobs and > _scheduler/docs on top of the already existing _active_tasks and just > polling _replicator db documents. >=20 > _scheduler/jobs is focused on replication jobs, their history, how > many times they started, stopped, their state. It returns replications > jobs indexed by their job ids, and includes jobs started from > _replicator dbs as well as from the _replicate HTTP endpoint. >=20 > _scheduler/docs in turn, is focused on showing the state of > replication documents. It returns replication jobs indexed by > _replicator db and doc id which started the jobs. >=20 > _scheduler/docs API was returning an "info" field for each job. That > info field was designed to contain details pertaining to the current > state of the job. So if there is an error it will have the error > message. For a completed job, it was showing the replication stats > persisted to the replication document, so that users do not have to > poll their status. >=20 > However, when the jobs were running or were in a pending state that > field was "null". The idea was to eventually replace "null" with stats > from _active_tasks such that users don't need to ever look at > _active_tasks to monitor their replication progress. So the proposal > here is to do exactly that. >=20 > To be specific: >=20 > 1) In the "pending" or "running" state _scheduler/docs will return > active_tasks status values in the "info" field. ("completed" is > already doing that btw). >=20 > 2) _scheduler/jobs will have a new "info" field which also show the > same status values >=20 > There is a draft PR for it >=20 > https://github.com/apache/couchdb/pull/2292#issuecomment-548526613 >=20 > Some examples of what it looks like: >=20 > https://github.com/apache/couchdb/pull/2292#issuecomment-548526346 > https://github.com/apache/couchdb/pull/2292#issuecomment-548526613 >=20 > What does everyone think? >=20 > Let's do a lazy consensus since we are replacing a place-holder null > value and adding a new field to _scheduler jobs. Let me know if this > is not appropriate and we need a different voting / discussion > procedure. >=20 > Cheers, > -Nick --=20 Professional Support for Apache CouchDB: https://neighbourhood.ie/couchdb-support/