couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <>
Subject Re: CouchDB 2.0: breaking the backward compatibility
Date Sat, 19 Jul 2014 09:50:08 GMT

On 19 Jul 2014, at 10:38 , Robert Samuel Newson <> wrote:

> I think this is backward. We are not proposing API changes "just because" BigCouch happens
to make them.
> Given that we have to bump the major version number, we are afforded an opportunity to
improve our API in significant ways for the first time since 1.0. We all know there are warts
to be fixed. The question is what to fix with 2.0, seeing as we’re making one.

That is what I meant to express, with the caveat that we should be
careful, taking a conservative stance, so we can meet in the middle.

> I would also hate to see low adoption of CouchDB 2.0 if we change too much, because I’ve
spent quite a bit of time trying to make it happen.
> Is there nothing, besides what comes with the BigCouch merge, that we wouldn’t want
to change for 2.0? At least the ability to add metadata without breaking 2.0 compliant clients
and libraries, I hope?

Most apps should continue to work on CouchDB 2.0.

Specifically, the regular document CRUD cycle should work as-is.
Especially moving things around in the JSON usually goes further
than the HTTP/Couch layer of most apps, as it is usually passed
down into the rest of the app, while HTTP specifics are kept on
the outside.

In that scenario, adding properties should be easier to do than
removing them (e.g. _conflicts could be standard, but renaming
_rev to _mvcc would break things more significantly), although
Bob mentioned the replicator compatibility as a major concern,
so we need to make sure this is doable.

My main point here is to start a discussion about how we would
go about evolving this down the road and my suggestion was the
separate API endpoint that we can mess with at will and gradually
introduce until we switch at a later time when we feel confident
that people have migrated, or a solid compatibility API is available.

I see us having three discussions:

1. What do we want to fix/break for 2.0?
2. How do we introduce fixes/breaks that we aren’t comfortable doing for 2.0?
3. What do we want to fix/break for later versions?

From this thread, I’d handwavingly suggest these fall into category 1:
(as per the “most apps should just continue to work”-mantra):

- timeout and heartbeat params for /_db_updates works in different way
then the same parameters for changes feed;
- we need to find the way to pass open_revs in POST body instead of
tweaking max URL param;
- we have /db/_revs_diff and /db/_revs_missing endpoints which are
doing the same job. Well, the latter is only used for pre-1.1 CouchDB
- /db/doc accepts conflicts, deleted_conflicts and revs params. In the
same time we provides meta one which includes each of specified.
- make eventsource feed to follow the specification format more better
then it does now
- MVCC for /db/_security and allow atomic changes for admins/members only
- a variant of “Changing the default respones for conflicts to include all
versions (or no version).” where ?conflicts=BOOL defaults to true, so we
get an additional _conflicts: [] member on regular GETs (if there are
conflicts), but not the conflicting versions themselves (see above note
about additional doc members)
- Fix the list API (inside couchjs) so that its a pure callback like
everything else.
- 'JSONP responses should be sent with a "application/javascript"'

These fall into category 3:

- Change _rev to _mvcc or other.
- Move document metadata elsewhere (sub-object, headers, whatever)
- Changing the default respones for conflicts to include all versions
(or no version).
- more RESTy API (move /_all_docs to /, db info to _info etc), self-defining REST API
- don’t pollute top level namespace (e.g. /database moves to /db/database)

This isn’t exhaustive, and we don’t yet know the answers to some of them.

As a repeat: with our new understanding of SemVer, we are free to ship CouchDB
3.0 a month after 2.0, if we really want to. We are not beholden to marketing
version numbers after 2.0 (strictly, we aren’t for 2.0 either, but it is
rather convenient :).

* * *

The view server protocol change suggested by Samuel is IMHO an internal
change that should not break BC unless people rely on implementation details.

* * *

Most apps should continue to work on CouchDB 2.0.


> Adding a new top-level _-prefixed field in couchdb causes the replicator to crash hard,
this is unacceptable brittle imo.

> B.
> On 18 Jul 2014, at 21:15, Jan Lehnardt <> wrote:
>> I’m major -1 on substantial API changes *just* because we are having some
>> by necessity of getting BigCouch in.
>> The minor improvements mentioned previously in this thread sound reasonable,
>> but changing the main JSON format seems like a rather bad idea as it will
>> just break all clients. While the scenario is a little different, I’d like to
>> avoid a Python 3 kind of situation (I think CouchDB 2.0 has more to offer over
>> 1.0 than Python 3 had over 2, but still, there is no need to make this harder
>> for our users, if we don’t have to).
>> That said, we likely want to evolve the API at some point and I think we should
>> nail down a strategy on *how* to do that, before getting into the details of
>> what should change.
>> One option, and I haven’t thought this through, would be to use separate ports
>> for a new API endpoint that we can evolve while keeping the current one. And
>> we can do the deprecation and switch dance some time in the future. We could
>> even try multiple competing APIs, even non HTTP APIs (all things, I’d love to
>> see, so we can learn from them). Of course there is a certain overhead in
>> maintaining this all, and I don’t know if there are any roadblocks in the way
>> BigCouch works today for implementing this.
>> Best
>> Jan
>> --
>> On 17 Jul 2014, at 21:03 , Russell Branca <> wrote:
>>> I would also love to see _rev renamed, and I think it's a good
>>> opportunity to flip around all the meta info as well. I'm still
>>> partial to moving the relevant metadata into the headers, and no
>>> longer including any _* fields in the doc, but I know there are
>>> proponents on both sides of the coin there. The most recent proposal I
>>> could find is to move all the metadata into a '_' field [1]. In 2.0 I
>>> would like to see us move all metadata into headers or into the '_'
>>> field, and rename 'rev'. There's a lot of code overlap for the two so
>>> it seems like an opportune time to do it.
>>> I wonder if it's reasonable to make the use of a '_' field or exposed
>>> through headers configurable. I'm not sure it's a great idea to do so,
>>> but it's at least worth thinking about.
>>> Exposing conflicts by default is another thing I'm keen on. The
>>> question is how to make it "fail" loudly so that client libraries
>>> don't just think it's the document body. An aggressive approach send a
>>> list of conflict revs rather than a doc object which will break all
>>> existing parsers and require users to deal with. Then if you want a
>>> particular rev, you'll need to specify it in the request.
>>> We could also cleanup the API endpoints to make them more RESTful. IMO
>>> things like _all_dbs and _all_docs should be the top level endpoints
>>> and the current info endpoints moved to _info or some such.
>>> Along the lines of API cleanup is the capabilities engine. I think
>>> this would be a great thing to land, and if done properly could be a
>>> self defining REST endpoint showing all the things the server is
>>> capable of and how to reach them.
>>> -Russell
>>> [1]
>>> On Thu, Jul 17, 2014 at 2:14 AM, Robert Samuel Newson
>>> <> wrote:
>>>> Great point, +1 to just making that change on master right now.
>>>> B.
>>>> On 16 Jul 2014, at 22:35, Robert Kowalski <> wrote:
>>>>> I would like to see 'JSONP responses should be sent with a
>>>>> "application/javascript"' (
>>>>> beside the two merges in the 2.0 release - it is a small, but breaking
>>>>> change and the original issue is flying around in Jira for years.
>>>>> Best,
>>>>> Robert
>>>>> 2014-07-13 22:17 GMT+02:00 Robert Samuel Newson <>:
>>>>>> Since we follow semantic versioning, the only meaning behind naming
>>>>>> next release 2.0 and not 1.7 is that it contains backwards incompatible
>>>>>> changes.
>>>>>> It’s for the CouchDB community as a whole to determine what is
and isn’t
>>>>>> in a release. Certainly merging in bigcouch and rcouch are a huge
part of
>>>>>> the 2.0 release, but they aren’t necessarily the only things. If
>>>>>> hadn’t changed the API in incompatible ways, they wouldn’t cause
a major
>>>>>> version bump.
>>>>>> With that said then, I’m interested in hearing what else, besides
the two
>>>>>> merges, we feel we want to take on in our first major revision bump
>>>>>> approximately forever? At minimum, I would like to see a change that
>>>>>> us to use versions of spidermonkey released after 1.8.5, whatever
>>>>>> change might be.
>>>>>> B.
>>>>>> On 13 Jul 2014, at 20:31, Joan Touzet <> wrote:
>>>>>>> Improving the view server protocol is a great idea, but it is
>>>>>>> for a 2.0 timeframe? I would think it would make more sense in
a 3.0
>>>>>>> timeframe, given 2.0 is all about merging forks, not writing
new features
>>>>>>> entirely from scratch.
>>>>>>> -Joan
>>>>>>> ----- Original Message -----
>>>>>>> From: "Robert Samuel Newson" <>
>>>>>>> To:
>>>>>>> Sent: Sunday, July 13, 2014 8:52:40 AM
>>>>>>> Subject: Re: CouchDB 2.0: breaking the backward compatibility
>>>>>>> Adding mvcc for _security is a great idea (happily, Cloudant
have done
>>>>>> so very recently, so I will be pulling that work over soon).
>>>>>>> A better view server protocol is also a great idea.
>>>>>>> On 13 Jul 2014, at 13:13, Samuel Williams <
>>>>>>> wrote:
>>>>>>>> On 13/07/14 23:47, Alexander Shorin wrote:
>>>>>>>>> Our view server is compiles functions on each view index
>>>>>>>>> instead of reusing inner cache. This is because of out-dated
>>>>>>>>> others design function are works differently from views.
While it's
>>>>>>>>> good to change and improve query server protocol completely,
this task
>>>>>>>>> requires more time to be done. We should have a least
plan B to do
>>>>>>>>> small steps in good direction.
>>>>>>>> As already suggested, here is my proposal for 2.0 view/query
>>>>>>>> I welcome people to suggest improvements/changes/ideas.
>>>>>>>> Kind regards,
>>>>>>>> Samuel

View raw message