couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Vatamaniuc <vatam...@apache.org>
Subject [DISCUSS] couch_event and FDB
Date Mon, 29 Apr 2019 18:29:06 GMT
After discussing how replicator might be implemented with fdb
https://lists.apache.org/thread.html/9338bd50f39d7fdec68d7ab2441c055c166041bd84b403644f662735@%3Cdev.couchdb.apache.org%3E,
and thinking about a global jobs queue for replicator (and indexing as
well, as Garren had mentioned), I had noticed that we might want to have
something like a couch_event subsystem. We'd need it that to drive creation
of replication jobs or pending indexing requests. So that's basically what
this discussion thread is about.

couch_event, for those who don't know, is a node-local event bus for
database-level events. When dbs are created, deleted, or updated it
publishes a corresponding event. Any interested listeners subscribe to
those events. Currently some of the listeners are:

 * ddoc cache : for ddoc updates, and db creates and deletes
 * ken : for ddoc updates to know when to start building indexes
 * _replicator db updates : to monitor changes to replication docs
 * _users db auth : to drive auth cache updates
 * _db_updates handler : drive db updates
 * smoosh : drive compaction triggers

This all happens in memory on each node. We use the nifty and fast khash
library to distribute events. If listeners register and then die, we
automatically clean that up.

Now with fdb, some of those users of couch_event go away (smoosh) but most
remain, and we need something to drive them.

There was a separate discussion for how to implement _db_updates handler,
which is a starting point for this one, but I had made a new thread as it
involves not just the _db_updates but those other things:

https://lists.apache.org/thread.html/a7bf140aea864286817bbd4f16f5a2f0a295f4d046950729400e0e2a@%3Cdev.couchdb.apache.org%3E

>From the _db_updates discussion I liked the proposal to use atomic ops to
accumulate and deduplicate events and consumers periodically reading and
resetting the stats.

The list of events currently being published is the following:

 * DbName, created
 * DbName, deleted
 * DbName, updated
 * DbName, local_updated (not needed anymore, used by smoosh only)
 * DbName, ddoc_updated
 * DbName, {ddoc_updated, DDocId}

(The {ddoc_updated, DDocId} makes it slightly more difficult as we'd need
to track specific DDocIDs. Maybe we could forgo such detailed updates and
let consumers keep track design documents on their own?)

But the idea is to have consumer worker processes and queues for each
consumer of the API. We could share them per-node. So, if on the same node
replicator and indexer want to find out about db update, we'd just add
extra callbacks for one, but they'd share the same consumer worker. Each
consumer would receive updates in their queue from the producers. Producers
would mostly end up doing atomic ops to update counts and periodically
monitor if consumer are still alive. Consumers would poll changes (or use
watches) to their update queues and notify their callbacks (start
replication jobs, update _db_updates DB, start indexing jobs, etc.).

Because consumers can die at any time, we don't want to have a growing
event queue, so each consumer will periodically report its health in the
db. The producer will monitor periodically all the consumers health
(asynchronously, using snapshot reads) and if consumers stop updating their
health their queues will be cleared. If they are alive next time they go to
read they'd have to re-register themselves in the consumers list. (This is
the same pattern from the replication proposal to replace process
linking/monitoring and cleanup that we have now).

The data model might look something like:

 ("couch_event", "consumers") = [C1, C2,...]
 ("couch_event", Ci, "heath") = (MaxTimeout, Timestamp)
 ("couch_event", Ci, "events", DbName, DbVer) = CreateDeleteUpdates
 ("couch_event", Ci, "events", DbName, DbVer, "ddoc_updates", DDocId) =
Updates

CreateDeleteUpdates is an integer that will encode create, delete, and
updates in one value using atomic ops:

* The value is initialized to 2, this is equivalent to "unknown" or "idle"
state.
* On a delete: min(1), so we reset the value down to 1
* On create: max(3) so we bring it up 3 (and if it was higher, it gets
reset to 3)
* On update: add(4)

The idea is that a consumer looking at the value periodically could figure
out if the database was deleted, created, created and updated, just
updated, or it was idle.

During each poll interval, for each database, the consumer would read the
value and reset it to 2 with a read conflict range on it. For ddocs it
would read the range and then clear it. If the value read was idle (2), it
will clear all the db event counter and ddoc range.

We could also enhance the consumers to accept only updates, and producers
would do filtering ahead of time. Db names for example could be
*/_replicator or */_users, docs would be "_design/*". Or that we only need
creates and deletes not updates. But that would mean not being able to
share consumer workers per node.

What does everyone think?

Cheers,
-Nick

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message