couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Miller <alexmil...@apple.com.INVALID>
Subject Re: [DISCUSS] : things we need to solve/decide : changes feed
Date Tue, 19 Mar 2019 01:03:03 GMT

> On Mar 5, 2019, at 4:04 PM, Adam Kocoloski <kocolosk@apache.org> wrote:
> With the incarnation and branch count in place we’d be looking at a design where the
KV pairs have the structure
> 
> (“changes”, Incarnation, Versionstamp) = (ValFomat, DocID, RevFormat, RevPosition,
RevHash, BranchCount)
> 
> where ValFormat is an enumeration enabling schema evolution of the value format in the
future, and RevFormat, RevPosition, RevHash are associated with the winning edit branch for
the document (not necessarily the edit that occurred at this version, matching current CouchDB
behavior) and carry the meanings defined in the revision storage RFC[2].



Do note that with versionstamped keys, and atomic operations in general, it’s important
to keep in mind that committing a transaction might return `commit_unknown_result`.  Transaction
loops will retry a `commit_unknown_result` error by default.  (Or, will, if your erlang/elixer
bindings copy the behavior of the rest of the bindings.)  So you’ll need some way of making
an insert into `changes` an idempotent operation.


I’ll volunteer three possible options:

1. The easiest case is if you happen to be inserting a known, fixed key (and preferably one
that contains a versionstamped value) in the same transaction as a versionstamped key, as
then you have a key to check in your database to tell if your commit happened or not.

2. If you’re doing an insert of just this key in a transaction, and your key space has relatively
infrequent writes, then you might be able to get away with remembering the initial read version
of your transaction, and issue a range scan from (“changes”, Incarnation, InitiailReadVersion)
-> (“changes”, infinity, infinity), and filter through looking for a value equal to
what you tried to write.

3. Accept that you might write duplicate values at different versionstamped keys, and write
your client code such that it will skip repeated values that it has already seen.

I had filed an internal bug long ago to complain about this before, which I’ve now copied
over to GitHub[1].  So if this becomes absurdly difficult to work around, feel free to show
up there to complain.

[1]: https://github.com/apple/foundationdb/issues/1321 <https://github.com/apple/foundationdb/issues/1321>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message