From dev-return-49192-archive-asf-public=cust-asf.ponee.io@couchdb.apache.org Tue Mar 24 14:32:34 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 8235A18065D for ; Tue, 24 Mar 2020 15:32:34 +0100 (CET) Received: (qmail 49456 invoked by uid 500); 24 Mar 2020 14:32:33 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 49432 invoked by uid 99); 24 Mar 2020 14:32:33 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Mar 2020 14:32:33 +0000 Received: from auth1-smtp.messagingengine.com (auth1-smtp.messagingengine.com [66.111.4.227]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 5EEBEF64 for ; Tue, 24 Mar 2020 14:32:33 +0000 (UTC) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailauth.nyi.internal (Postfix) with ESMTP id 2B51E27C0054 for ; Tue, 24 Mar 2020 10:32:33 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Tue, 24 Mar 2020 10:32:33 -0400 X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrudehuddgfeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucgoufhprghmkfhpqdhouhhtucdlhedttddmnecujf gurhephfgtgfgguffffhfvjgfkofesthhqmhdthhdtvdenucfhrhhomheptfhosggvrhht ucfurghmuhgvlhcupfgvfihsohhnuceorhhnvgifshhonhesrghprggthhgvrdhorhhgqe enucfkphepkeelrddvfeekrdduheegrddvgeelnecuvehluhhsthgvrhfuihiivgeptden ucfrrghrrghmpehmrghilhhfrhhomheprhhnvgifshhonhdomhgvshhmthhprghuthhhph gvrhhsohhnrghlihhthidqleefgedvtddvjedvqdduudelgeejtdejjedqrhhnvgifshho nheppegrphgrtghhvgdrohhrghesfhgrshhtmhgrihhlrdhfmh X-ME-Proxy: Received: from [10.73.10.6] (unknown [89.238.154.249]) by mail.messagingengine.com (Postfix) with ESMTPA id C13213280067 for ; Tue, 24 Mar 2020 10:32:32 -0400 (EDT) From: Robert Samuel Newson Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.60.0.2.5\)) Subject: Re: [DISCUSS] Mango indexes on FDB Date: Tue, 24 Mar 2020 14:32:29 +0000 References: <4ebfca4c-5537-49d6-8e2c-6ea4a98ca69b@www.fastmail.com> <83da7901-6baa-f153-1bff-d60877eee403@apache.org> To: CouchDB Developers In-Reply-To: Message-Id: <518C26B2-0CD2-4FDA-84D1-EDD346A2DBFA@apache.org> X-Mailer: Apple Mail (2.3608.60.0.2.5) Hi, We had a long discussion on the CouchDB Slack on this topic, which I = will brutally summarise; While we intend to update json indexes in the same transaction as the = associated document update, there is a concern that this won't always be = possible (large document and large number of indexes to update at once) = and that it will not apply to javascript map indexes or search indexes. It was therefore felt that having an immediate "Not ready" signal for = just _some_ calls to _find, based on the type of backing index, was a = bad and confusing api. We also discussed _find calls where the user does not specify an index, = and concluded that we would be free to choose between using the = _all_docs index (which is always up to date but rarely the best index = for a given selector) or blocking to update a better but stale index. Summary-ing my summarisation; 1) if you specify an index, we'll use it even if we have to update it, = no matter how long that takes. 2) if you don't specify an index, it's the dealers choice. The details = here may change in point releases. No new status code is therefore needed. B. > On 24 Mar 2020, at 12:51, Garren Smith wrote: >=20 > On Tue, Mar 24, 2020 at 1:30 AM Joan Touzet wrote: >=20 >>=20 >>=20 >> On 2020-03-23 4:46 p.m., Mike Rhodes wrote: >>> Garren, >>>=20 >>> Very much +1 on this suggestion, as it is, at least for me, what I'd >> expect to happen if I were leaving the system to select an index -- = as you >> imply, the build process almost certainly takes longer than using the >> _all_docs index. In addition, for the common case where there is a = less >> optimal but still useful index available, one might expect that index = to be >> used in preference to the "better" but unbuilt one. >>=20 >> I agree. >>=20 >>> But I do think this is important: >>>=20 >>>> We can amend the warning message >>>> to let them know that they have an index that is building that = could >>>> service the index when it's ready. >>>=20 >>> Otherwise it's a bit too easy to get confused when trying to = understand >> the reason why an index you were _sure_ should've been used in fact = was not. >>=20 >> Question: Imagine a node that's been offline for a bit and is just >> coming back on. (I'm not 100% sure how this works in FDB land.) If >> there's a (stale) index on disk, and the index is being updated, and = the >> index on disk is kind of stale...what happens? >>=20 >=20 > With couchdb_layer this can't happen as each CouchDB node is stateless = and > doesn't actually keep any indexes. Everything would be in = FoundationDB. So > if the index is built then it is built and ready for all couch_layer = nodes. >=20 > FoundationDB storage servers could fall behind the Tlogs. I'm not 100% = sure > what would happen in this case. But it would be consistent for all > couch_layer nodes. >=20 >=20 >=20 >>=20 >> -Joan