couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: 2.0 Progress
Date Mon, 29 Jun 2015 20:12:30 GMT
Heya,

so, confession time!

A few of us got impatient about 2.0 progress so we started a little skunk work group to go
through the remaining issues. It’s time to open this up to the rest of the community (because
we need your help, and it’s only fair :)

The list that was shared in this thread was a first result from a lengthy IRC meeting. We’ve
worked through a few more things, some of which you’ve already seen landing in the respective
modules.

There are a few more todos listed in our notes: https://pad.apache.org/p/couchdb-2.0-sprint
(requires ASF login, will post full contents at the bottom) and of course all 2.0-tagged issues
on JIRA and our open Pull Requests.

Our goal was to get a first feature complete version of CouchDB 2.0 out as a Release Candidate
or Beta release soon-ish.

We are currently waiting for dynamic URL route configuration (see notes for details) from
Cloudant and the setup wizard work from Robert Kowalski and a few odds and ends.

I’d like to get an RC1/Beta1 released shortly after these things land, so we can start to
invite the larger community to test and report issues.

What do you think?

Best
Jan
--
June 28:
    ## diff to 2.0
    i think the log format of chttpd vs backdoorport is different:
        https://github.com/apache/couchdb-couch/blob/master/src/couch_httpd.erl#L614
        https://github.com/apache/couchdb-chttpd/blob/e7f9ed83da8e77aeaf8b140344280e990b0580ad/src/chttpd.erl#L262
sounds trivial but a lot of folks have tooling for parsing their logs for stats etc in place
(maybe readd the both "-")


June 27:

## Personas:

1. Person who installs CouchDB on one or more machines (admin)
2. Person who administers a CouchDB installation (admin)
3. Person who develops code/apps that runs against CouchDB (dev)
4. Person that uses the code/apps from 3. to make requests against CouchDB (user)
5. Person that teaches about CouchDB (e.g. helping others to build apps, consulting, running
workshops) (teacher)






June 26

broken in 2.0:
    End-point /db/_compact/design-doc is broken - https://issues.apache.org/jira/browse/COUCHDB-2729

June 25:


# Missing in 2.x:
    - make fabric timeouts configurable: https://github.com/apache/couchdb-fabric/blob/19df05419048478a92c747cb5c20c0c7a0331f51/src/fabric_util.erl#L138


## Available in 1.x but missing in 2.x:

 - bulk docs, all_or_nothing: does not “either all documents are written to the database,
or no documents are written to the database, in the event of a system failure during commit.”
-- maybe remove it?
 - _temp_view / temporary views -- had a good discussion some time ago with paul davis about
that how it could be implemented

### fix make havascript:
    - couchdb-harness can’t be used because they deleted some couch-only tests instead of
skipping them
    - test/javascript/tests/* all need to be updated to accommodate cluster API sema

June 24:

- [ ] fix `make javascript (Jan, Alexander)
  - maybe poach PouchDB’s couchdb-test-harness

- [ ] merge _bulk_revs (Alexander)
    - https://github.com/apache/couchdb-chttpd/pull/33

= [ ] fix writing .uri file


June 19th:


# Missing Features

## Available in 1.x but missing in 2.x

_utils is missing in 2.x





June 18th:

# QA

## Continuous Integration

All TBD:
- start a recruiting drive to find people who want to own this
- start discussions with asf infra on what a good setup would look like, and then work towards
it




# Missing Features

(continued from June 17th)


## Availble in 1.x but missing in 2.x

Updates on _node_config and _node_stats

We will now do _node/<fqdn>/_config and _node/<fqdn>/_stats with the option of
adding more endpoints there, later.

The rest stays as is.



## Missing in 2.x

- cluster rebalance
  - Cloudant might be able to open source a `rebal` tool






June 17th:

# Missing Features

## Availble in 1.x but missing in 2.x
- [X] vhosts: Done.

- [X ] /_config on :5984
  - no cluster wide config
  - per-node config can be set from any node:
  - only expose /_node_config/<node-fqdn>/<section>/<key>
    - first stab of this already in master
  - Fauxton to offer “tabbed” interface for per-cluster config view
  - post 2.0, if we find a backing store that would give us a cluster-consistent :5984/_config/section/key,
we can enable that.


- [X]: /_stats on :5984
  - same as with /_config:
      - /_node_stats/<fqdn>/<etc>



- [X] dynamic http endpoint configuration for :5984
  - integrate Cloudant’s work that allows for erlang applications to register their own
routes in a .priv file. Adding a route is done by adding an app, changing a route is by updating
an app and its .priv file
  - /_config can be used to disable endpoints
  - TBD: what if two applications define the same endpoint? Options:
      - undefined
      - offending app doesn’t load
      - server doesn’t start / stops
      - define that endpoints can’t be overwritten, define strict sort order for loading,
load core apps preferred to ensure working node + *handwaving*
 

- [X] externals
- [X] proxy
 - keep both for 2.0
 - add to chttpd routes loading:
   - 1. load all application routes
   - 2. load [external] and [proxy] routes
   - 3. listen to [external] and [proxy] config changes and adjust route map at runtime
 

- [X] logging configuration of level & file & apps (http://docs.couchdb.org/en/latest/config/logging.html)
  - dynamic configuration of the log file won’t be possible anymore, because lager works
differently
  - since the configuration for lager is more comprehensive, 1.x-style configuration is no
longer possible
  -TBD:  however, it is possible to set the log level at runtime. Two options:
	• -  setting /_config/log:backend/level to <logatom> / couch_log listens to config
updates and calls https://github.com/basho/lager#runtime-loglevel-changes
	• - new endpoint, e.g. PUT /_log/backend/<logatom>
  - log atoms change, mark as BC break


- [X] rewrites (?)
  - keep, should just work


- [X] documentation
  - we have (or will have soon): API docs, upgrade/migration guides / new features
  - we *don’t* have: how the cluster works (dynamo + couch specific details)
    - needs help!


- [X ] build script
  - finish unix version (Jan)
  - integrate with Mac app (Jan)
  - TBD: Windows
	• - Can Nick North help?
	• - maybe ship 2.0 with experimental Windows, since the code has never run on Windows(?)

## Missing in 2.x

- [X] single node mode
  - “single node mode” just means “cluster of one”
  - the chttp-layer might be slower than couch_httpd, if so, we have good incentives to make
this fast.
  - going from 1.x to 2.x-single-node will requires changes in how an app uses the CouchDB
API (as per our Breaking Changes documentation).
  - :5986 is 127.0.0.1-only and strongly discouraged for general use, ops folks might want
to use it.


- [X] add/remove nodes, cluster rebalance
  - add/remove node needs documentation
  - rebalancing:
       - tool to help figure out what needs doing
       - might be ready by 2.0, post 2.0 otherwise



> On 17 Jun 2015, at 17:50, Jan Lehnardt <jan@apache.org> wrote:
> 
> Hey all,
> 
> Alexander, Bob and I had a bit of a brainstorming session today on what is missing to
get 2.0 out the door. We talked about missing features and what to do about them. The following
notes is what we think is best, but of course, these are just suggestions and we’d love
your feedback! Especially on the TBD items.
> 
> The notes are dense, let us know if you have any questions or need any clarification
:)
> 
> * * *
> 
> # Missing Features
> 
> ## Availble in 1.x but missing in 2.x
> 
> - [X] vhosts: Done.
> 
> - [X] /_config on :5984
>  - no cluster wide config
>  - per-node config can be set from any node:
>  - only expose /_node_config/<node-fqdn>/<section>/<key>
>  - first stab of this already in master
>  - Fauxton to offer “tabbed” interface for per-cluster config view
>  - post 2.0, if we find a backing store that would give us a cluster-consistent :5984/_config/section/key,
we can enable that.
> 
> 
> - [X]: /_stats on :5984
>  - same as with /_config:
>      - /_node_stats/<fqdn>/<etc>
> 
> 
> - [X] dynamic http endpoint configuration for :5984
>  - integrate Cloudant’s work that allows for erlang applications to register their
own routes in a priv/ file. Adding a route is done by adding an app, changing a route is by
updating an app and its priv/ file
>  - /_config can be used to disable endpoints
>  - TBD: what if two applications define the same endpoint? Options:
>      - undefined
>      - offending app doesn’t load
>      - server doesn’t start / stops
>      - define that endpoints can’t be overwritten, define strict sort order for loading,
load core apps preferred to ensure working node + *handwaving*
> 
> 
> - [X] externals & proxy
> - keep both for 2.0
> - add to chttpd routes loading:
>   - 1. load all application routes
>   - 2. load [external] and [proxy] routes
>   - 3. listen to [external] and [proxy] config changes and adjust route map at runtime
> 
> 
> - [X] logging configuration of level & file & apps (http://docs.couchdb.org/en/latest/config/logging.html)
>  - dynamic configuration of the log file won’t be possible anymore, because lager works
differently
>  - since the configuration for lager is more comprehensive, 1.x-style configuration is
no longer possible
>  - TBD: however, it is possible to set the log level at runtime. Two options:
> 	-  setting /_config/log:backend/level to <logatom> / couch_log listens to config
updates and calls https://github.com/basho/lager#runtime-loglevel-changes
> 	- new endpoint, e.g. PUT /_log/backend/<logatom>
>  - log atoms change, mark as BC break
> 
> 
> - [X] rewrites (?)
>  - keep, should just work
> 
> 
> - [X] documentation
>  - we have (or will have soon): API docs, upgrade/migration guides / new features
>  - we *don’t* have: how the cluster works (dynamo + couch specific details)
>    - needs help!
> 
> - [X ] build script
>  - finish unix version (Jan)
>  - integrate with Mac app (Jan)
>  - TBD: Windows
> 	- Can Nick North help?
> 	- maybe ship 2.0 with experimental Windows, since the code has never run on Windows(?)
> 
> 
> ## Missing in 2.x
> 
> - [X] single node mode
>  - “single node mode” just means “cluster of one”
>  - the chttp-layer might be slower than couch_httpd, if so, we have good incentives to
make this fast.
>  - going from 1.x to 2.x-single-node will requires changes in how an app uses the CouchDB
API (as per our Breaking Changes documentation).
>  - :5986 is 127.0.0.1-only and strongly discouraged for general use, ops folks might
want to use it.
> 
> 
> - [X] add/remove nodes, cluster rebalance
>  - add/remove node needs documentation
>  - rebalancing:
>       - tool to help figure out what needs doing
>       - might be ready by 2.0, post 2.0 otherwise
> 
> * * *
> 
> Best
> Jan
> -- 
> 

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/


Mime
View raw message