couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Garren Smith <gar...@apache.org>
Subject Design doc index switching
Date Thu, 16 May 2019 06:51:28 GMT
Hi Everyone,

A common pattern we see for updating large indexes that can take a few days
to build, is create a new design docs with the new updated views. Then once
the new design doc is built, a user changes the new design doc’s id to the
old design doc. That way the CouchDB url for the views remain the same and
any requests to the design doc url automatically get the latest views only
once they built.

This is an effective way of managing building large indexes, but the
process is quite complicated and often users get it wrong. I would like to
propose that we move this process into CouchDB and let CouchDB handle the
actual process. From a users perspective, they would add a field to the
options of a design document that lets CouchDB know, that this build needs
to be built in the background and only replace the current index once its
built:

```
{
  "_id": "_design/design-doc-id",
  "_rev": "2-8d361a23b4cb8e213f0868ea3d2742c2",
  "views": {
    "map-view": {
      "map": "function (doc) {\n  emit(doc._id, 1);\n}"
    }
  },
  "language": "javascript",
    "options": {
        "build_and_replace": true
    }
}
```

I think this is something we could build quite effectively once we have
CouchDB running on top of FoundationDB. I don’t want to implement it for
version 1 of CouchDB on FDB, but it would be nice to keep this in mind as
we build out the map/reduce indexes.

What do you think? Any issues we might have by doing this internally?

Cheers
Garren

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message