couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: [DISCUSS] couchdb 4.0 transactional semantics
Date Tue, 21 Jul 2020 15:42:33 GMT


> On 21. Jul 2020, at 17:24, Bessenyei Balázs Donát <bessbd@apache.org> wrote:
> 
> I think being able to leverage FoundationDB's serializability is an
> awesome idea! +1 (non-binding) on all 4 points.
> I also support the idea of changing the API in backwards-incompatible
> ways if that makes things more convenient / streamlined. I wonder,
> does this mean other, backwards-incompatible changes are also welcome
> in the next major? (Given that replicator-compatibility (from later on
> in this thread) is expected.)

We rather don’t like to break things just because we can :)

Do you have anything specific in mind?

Best
Jan
—

> 
> 
> Thank you,
> 
> Donat
> 
> 
> On Thu, 16 Jul 2020 at 18:26, Paul Davis <paul.joseph.davis@gmail.com> wrote:
>> 
>> From what I'm reading it sounds like we have general consensus on a few things:
>> 
>> 1. A single CouchDB API call should map to a single FDB transaction
>> 2. We absolutely do not want to return a valid JSON response to any
>> streaming API that hit a transaction boundary (because data
>> loss/corruption)
>> 3. We're willing to change the API requirements so that 2 is not an issue.
>> 4. None of this applies to continuous changes since that API call was
>> never a single snapshot.
>> 
>> If everyone generally agrees with that summarization, my suggestion
>> would be that we just revisit the new pagination APIs and make them
>> the only behavior rather than having them be opt-in. I believe those
>> APIs already address all the concerns in this thread and the only
>> reason we kept the older versions with `restart_tx` was to maintain
>> API backwards compatibility at the expense of a slight change to
>> semantics of snapshots. However, if there's a consensus that the
>> semantics are more important than allowing a blanket `GET
>> /db/_all_docs` I think it'd make the most sense to just embrace the
>> pagination APIs that already exist and were written to cover these
>> issues.
>> 
>> The only thing I'm not 100% on is how to deal with non-continuous
>> replications. I.e., the older single shot replication. Do we go back
>> with patches to older replicators to allow 4.0 compatibility? Just
>> declare that you have to mediate a replication on the newer of the two
>> CouchDB deployments? Sniff the replicator's UserAgent and behave
>> differently on 4.x for just that special case?
>> 
>> Paul
>> 
>> On Wed, Jul 15, 2020 at 7:25 PM Adam Kocoloski <kocolosk@apache.org> wrote:
>>> 
>>> Sorry, I also missed that you quoted this specific bit about eagerly requesting
a new snapshot. Currently the code will just react to the transaction expiring, then wait
till it acquires a new snapshot if “restart_tx” is set (which can take a couple of milliseconds
on a FoundationDB cluster that is deployed across multiple AZs in a cloud Region) and then
proceed.
>>> 
>>> Adam
>>> 
>>>> On Jul 15, 2020, at 6:54 PM, Adam Kocoloski <kocolosk@apache.org> wrote:
>>>> 
>>>> Right now the code has an internal “restart_tx” flag that is used to
automatically request a new snapshot if the original one expires and continue streaming the
response. It can be used for all manner of multi-row responses, not just _changes.
>>>> 
>>>> As this is a pretty big change to the isolation guarantees provided by the
database Bob volunteered to elevate the issue to the mailing list for a deeper discussion.
>>>> 
>>>> Cheers, Adam
>>>> 
>>>>> On Jul 15, 2020, at 11:38 AM, Joan Touzet <wohali@apache.org> wrote:
>>>>> 
>>>>> I'm having trouble following the thread...
>>>>> 
>>>>> On 14/07/2020 14:56, Adam Kocoloski wrote:
>>>>>> For cases where you’re not concerned about the snapshot isolation
(e.g. streaming an entire _changes feed), there is a small performance benefit to requesting
a new FDB transaction asynchronously before the old one actually times out and swapping over
to it. That’s a pattern I’ve seen in other FDB layers but I’m not sure we’ve used
it anywhere in CouchDB yet.
>>>>> 
>>>>> How does _changes work right now in the proposed 4.0 code?
>>>>> 
>>>>> -Joan
>>>> 
>>> 


Mime
View raw message