couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Samuel Newson <rnew...@apache.org>
Subject Re: CouchDB 2.0: breaking the backward compatibility
Date Sat, 19 Jul 2014 10:27:18 GMT

I agree with you on the category split and, obviously, that we can make a 3.0 whenever we
like, since it only indicates compatibility breaks.

MVCC for _security will be great since it will enable us to heal _security writes during partitions
with the same logic we use for documents. Cloudant worked around (hacked around) that problem
for a long time but we’ve finally done the work to add MVCC for reals, so it’ll be nice
to bring that enhancement to CouchDB officially.

I don’t think defaulting to conflicts=true is quite the right change. I was thinking that
conflicted document would return a 300 Multiple Choices instead of a 200 OK (the response
body format TBD but imagine a JSON array of each conflicting leaf revision). I agree that
a change like that will break every client, but that would be the intention. A CouchDB 3.0
would be much more forthcoming about its fundamental architecture and would largely eschew
the attempts to present an arbitrary "winning" revision.

One further thought occurs, could we totally ditch the code that returns "text/plain" content-type?
All that "are you a browser?" logic? It made sense at the time, but I feel it confuses more
than it helps today.

B.


On 19 Jul 2014, at 10:50, Jan Lehnardt <jan@apache.org> wrote:

> 
> On 19 Jul 2014, at 10:38 , Robert Samuel Newson <rnewson@apache.org> wrote:
> 
>> 
>> I think this is backward. We are not proposing API changes "just because" BigCouch
happens to make them.
>> 
>> Given that we have to bump the major version number, we are afforded an opportunity
to improve our API in significant ways for the first time since 1.0. We all know there are
warts to be fixed. The question is what to fix with 2.0, seeing as we’re making one.
> 
> That is what I meant to express, with the caveat that we should be
> careful, taking a conservative stance, so we can meet in the middle.
> 
> 
>> I would also hate to see low adoption of CouchDB 2.0 if we change too much, because
I’ve spent quite a bit of time trying to make it happen.
>> 
>> Is there nothing, besides what comes with the BigCouch merge, that we wouldn’t
want to change for 2.0? At least the ability to add metadata without breaking 2.0 compliant
clients and libraries, I hope?
> 
> 
> Most apps should continue to work on CouchDB 2.0.
> 
> Specifically, the regular document CRUD cycle should work as-is.
> Especially moving things around in the JSON usually goes further
> than the HTTP/Couch layer of most apps, as it is usually passed
> down into the rest of the app, while HTTP specifics are kept on
> the outside.
> 
> In that scenario, adding properties should be easier to do than
> removing them (e.g. _conflicts could be standard, but renaming
> _rev to _mvcc would break things more significantly), although
> Bob mentioned the replicator compatibility as a major concern,
> so we need to make sure this is doable.
> 
> My main point here is to start a discussion about how we would
> go about evolving this down the road and my suggestion was the
> separate API endpoint that we can mess with at will and gradually
> introduce until we switch at a later time when we feel confident
> that people have migrated, or a solid compatibility API is available.
> 
> I see us having three discussions:
> 
> 1. What do we want to fix/break for 2.0?
> 2. How do we introduce fixes/breaks that we aren’t comfortable doing for 2.0?
> 3. What do we want to fix/break for later versions?
> 
> 
> From this thread, I’d handwavingly suggest these fall into category 1:
> (as per the “most apps should just continue to work”-mantra):
> 
> - timeout and heartbeat params for /_db_updates works in different way
> then the same parameters for changes feed;
> - we need to find the way to pass open_revs in POST body instead of
> tweaking max URL param;
> - we have /db/_revs_diff and /db/_revs_missing endpoints which are
> doing the same job. Well, the latter is only used for pre-1.1 CouchDB
> replicator.
> - /db/doc accepts conflicts, deleted_conflicts and revs params. In the
> same time we provides meta one which includes each of specified.
> - make eventsource feed to follow the specification format more better
> then it does now
> - MVCC for /db/_security and allow atomic changes for admins/members only
> - a variant of “Changing the default respones for conflicts to include all
> versions (or no version).” where ?conflicts=BOOL defaults to true, so we
> get an additional _conflicts: [] member on regular GETs (if there are
> conflicts), but not the conflicting versions themselves (see above note
> about additional doc members)
> - Fix the list API (inside couchjs) so that its a pure callback like
> everything else.
> - 'JSONP responses should be sent with a "application/javascript"'
> 
> 
> These fall into category 3:
> 
> - Change _rev to _mvcc or other.
> - Move document metadata elsewhere (sub-object, headers, whatever)
> - Changing the default respones for conflicts to include all versions
> (or no version).
> - more RESTy API (move /_all_docs to /, db info to _info etc), self-defining REST API
> - don’t pollute top level namespace (e.g. /database moves to /db/database)
> 
> This isn’t exhaustive, and we don’t yet know the answers to some of them.
> 
> As a repeat: with our new understanding of SemVer, we are free to ship CouchDB
> 3.0 a month after 2.0, if we really want to. We are not beholden to marketing
> version numbers after 2.0 (strictly, we aren’t for 2.0 either, but it is
> rather convenient :).
> 
> * * *
> 
> The view server protocol change suggested by Samuel is IMHO an internal
> change that should not break BC unless people rely on implementation details.
> 
> 
> * * *
> 
> Most apps should continue to work on CouchDB 2.0.
> 
> Jan
> -- 
> 
> 
> 
> 
> 
> 
> 
> 
>> Adding a new top-level _-prefixed field in couchdb causes the replicator to crash
hard, this is unacceptable brittle imo.
> 
>> 
>> B.
>> 
>> On 18 Jul 2014, at 21:15, Jan Lehnardt <jan@apache.org> wrote:
>> 
>>> I’m major -1 on substantial API changes *just* because we are having some
>>> by necessity of getting BigCouch in.
>>> 
>>> The minor improvements mentioned previously in this thread sound reasonable,
>>> but changing the main JSON format seems like a rather bad idea as it will
>>> just break all clients. While the scenario is a little different, I’d like
to
>>> avoid a Python 3 kind of situation (I think CouchDB 2.0 has more to offer over
>>> 1.0 than Python 3 had over 2, but still, there is no need to make this harder
>>> for our users, if we don’t have to).
>>> 
>>> That said, we likely want to evolve the API at some point and I think we should
>>> nail down a strategy on *how* to do that, before getting into the details of
>>> what should change.
>>> 
>>> One option, and I haven’t thought this through, would be to use separate ports
>>> for a new API endpoint that we can evolve while keeping the current one. And
>>> we can do the deprecation and switch dance some time in the future. We could
>>> even try multiple competing APIs, even non HTTP APIs (all things, I’d love
to
>>> see, so we can learn from them). Of course there is a certain overhead in
>>> maintaining this all, and I don’t know if there are any roadblocks in the way
>>> BigCouch works today for implementing this.
>>> 
>>> Best
>>> Jan
>>> --
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 17 Jul 2014, at 21:03 , Russell Branca <chewbranca@apache.org> wrote:
>>> 
>>>> I would also love to see _rev renamed, and I think it's a good
>>>> opportunity to flip around all the meta info as well. I'm still
>>>> partial to moving the relevant metadata into the headers, and no
>>>> longer including any _* fields in the doc, but I know there are
>>>> proponents on both sides of the coin there. The most recent proposal I
>>>> could find is to move all the metadata into a '_' field [1]. In 2.0 I
>>>> would like to see us move all metadata into headers or into the '_'
>>>> field, and rename 'rev'. There's a lot of code overlap for the two so
>>>> it seems like an opportune time to do it.
>>>> 
>>>> I wonder if it's reasonable to make the use of a '_' field or exposed
>>>> through headers configurable. I'm not sure it's a great idea to do so,
>>>> but it's at least worth thinking about.
>>>> 
>>>> Exposing conflicts by default is another thing I'm keen on. The
>>>> question is how to make it "fail" loudly so that client libraries
>>>> don't just think it's the document body. An aggressive approach send a
>>>> list of conflict revs rather than a doc object which will break all
>>>> existing parsers and require users to deal with. Then if you want a
>>>> particular rev, you'll need to specify it in the request.
>>>> 
>>>> We could also cleanup the API endpoints to make them more RESTful. IMO
>>>> things like _all_dbs and _all_docs should be the top level endpoints
>>>> and the current info endpoints moved to _info or some such.
>>>> 
>>>> Along the lines of API cleanup is the capabilities engine. I think
>>>> this would be a great thing to land, and if done properly could be a
>>>> self defining REST endpoint showing all the things the server is
>>>> capable of and how to reach them.
>>>> 
>>>> 
>>>> 
>>>> -Russell
>>>> 
>>>> 
>>>> [1] http://mail-archives.apache.org/mod_mbox/couchdb-dev/201312.mbox/%3C529DE44C.4090400@bigbluehat.com%3E
>>>> 
>>>> On Thu, Jul 17, 2014 at 2:14 AM, Robert Samuel Newson
>>>> <rnewson@apache.org> wrote:
>>>>> Great point, +1 to just making that change on master right now.
>>>>> 
>>>>> B.
>>>>> 
>>>>> On 16 Jul 2014, at 22:35, Robert Kowalski <rok@kowalski.gd> wrote:
>>>>> 
>>>>>> I would like to see 'JSONP responses should be sent with a
>>>>>> "application/javascript"' (https://github.com/apache/couchdb/pull/236)
>>>>>> beside the two merges in the 2.0 release - it is a small, but breaking
>>>>>> change and the original issue is flying around in Jira for years.
>>>>>> 
>>>>>> Best,
>>>>>> Robert
>>>>>> 
>>>>>> 
>>>>>> 2014-07-13 22:17 GMT+02:00 Robert Samuel Newson <rnewson@apache.org>:
>>>>>> 
>>>>>>> 
>>>>>>> Since we follow semantic versioning, the only meaning behind
naming our
>>>>>>> next release 2.0 and not 1.7 is that it contains backwards incompatible
>>>>>>> changes.
>>>>>>> 
>>>>>>> It’s for the CouchDB community as a whole to determine what
is and isn’t
>>>>>>> in a release. Certainly merging in bigcouch and rcouch are a
huge part of
>>>>>>> the 2.0 release, but they aren’t necessarily the only things.
If they
>>>>>>> hadn’t changed the API in incompatible ways, they wouldn’t
cause a major
>>>>>>> version bump.
>>>>>>> 
>>>>>>> With that said then, I’m interested in hearing what else, besides
the two
>>>>>>> merges, we feel we want to take on in our first major revision
bump in
>>>>>>> approximately forever? At minimum, I would like to see a change
that allows
>>>>>>> us to use versions of spidermonkey released after 1.8.5, whatever
that
>>>>>>> change might be.
>>>>>>> 
>>>>>>> B.
>>>>>>> 
>>>>>>> On 13 Jul 2014, at 20:31, Joan Touzet <wohali@apache.org>
wrote:
>>>>>>> 
>>>>>>>> Improving the view server protocol is a great idea, but it
is appropriate
>>>>>>>> for a 2.0 timeframe? I would think it would make more sense
in a 3.0
>>>>>>>> timeframe, given 2.0 is all about merging forks, not writing
new features
>>>>>>>> entirely from scratch.
>>>>>>>> 
>>>>>>>> -Joan
>>>>>>>> 
>>>>>>>> ----- Original Message -----
>>>>>>>> From: "Robert Samuel Newson" <rnewson@apache.org>
>>>>>>>> To: dev@couchdb.apache.org
>>>>>>>> Sent: Sunday, July 13, 2014 8:52:40 AM
>>>>>>>> Subject: Re: CouchDB 2.0: breaking the backward compatibility
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Adding mvcc for _security is a great idea (happily, Cloudant
have done
>>>>>>> so very recently, so I will be pulling that work over soon).
>>>>>>>> 
>>>>>>>> A better view server protocol is also a great idea.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 13 Jul 2014, at 13:13, Samuel Williams <
>>>>>>> space.ship.traveller@gmail.com> wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 13/07/14 23:47, Alexander Shorin wrote:
>>>>>>>>>> Our view server is compiles functions on each view
index update
>>>>>>>>>> instead of reusing inner cache. This is because of
out-dated protocol:
>>>>>>>>>> others design function are works differently from
views. While it's
>>>>>>>>>> good to change and improve query server protocol
completely, this task
>>>>>>>>>> requires more time to be done. We should have a least
plan B to do
>>>>>>>>>> small steps in good direction.
>>>>>>>>> As already suggested, here is my proposal for 2.0 view/query
server:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> https://docs.google.com/document/d/1JtfvCpNB9pRQyLhS5KkkEdJ-ghSCv89xnw5HDMTCsp8/edit
>>>>>>>>> 
>>>>>>>>> I welcome people to suggest improvements/changes/ideas.
>>>>>>>>> 
>>>>>>>>> Kind regards,
>>>>>>>>> Samuel
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> 
> 


Mime
View raw message