couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Chesneau <bchesn...@gmail.com>
Subject rcouch merge status
Date Wed, 12 Feb 2014 13:11:50 GMT
Hi all,

Sorry for this late status. I have been busy with a cold and needed work
on the code.

Anyway here is a quick status of what  have been done since the last
status:

1) The view changes has been merged. It includes some code from what
have beem given last year to the Apache foundation + unitests and a
small refactoring that was already in action in other places I manage.

The view change feature include:

- index all changes in an index by sequences and by key: it's possible
  to get all the changes of an index from a sequence or all changes for
a key or in a range since a a sequence. Removed key are also returned
which is new compared to the current code in the rcouch core core. (this
new feature can also be useful later for someone that would like to
build a feature chaining map/reduces steps eventually).
- api to listen on index changes. You can retrieve all changes at once,
  listen until a change happen (~= longpoll), listen continuously on all
changes. Unlike the current changes internal api, this api is designed
so it can be reused by any Erlang plugin the erlang way.
- api to retrieve changes of a db using a view. If the indexation of
  changes is enabled in a design doc, it will be used in place of the
current view filter. If the indexation is not enabled the old and non
efficient view filter is used.
- Add a facility to index the views in a continuous manner. While this
  api is designed internally to be reused, it's only enabled for now ehn
listening on _changes. or by enable it in the change listener described
above. When enabled the view index is refreshed evey N secs (can be set
in the config file) or after N updates in the database (can be set in
the config file).
- The replication has also been been modified so you can replicate using
  a view index as a filte. (And yes you can filter the changes using a
key which is can give you a feature similar to the channels in the
couchbase sync gateway)
- the views changes has now unitests.

2) couch_log has been improved and is now using lager [1]. With this change
instead to use our own way to log file we are using lager which improve
the logging and  will make couchdb more tolerant in the face of large or
many log messages, it won't out of memory the node.

Note: Lager can handle multiple backend but we are for now only handling
the file and console backend in the ini file. Other backends can be
configured using the app.config file. Also the code is slightly
different from the current one in rcouch and imported from a private
branch in use somewhere: it allows to update the log configuration not
only in the app.config but also in the ini and handle the log level /
module.


What is missing from rcouch:

- validate doc on read [2]
-  Write only databases [3]
- Filter fields in included doc when returned from a _changes [4]
- optimisations [5]

Optimisations may be wait the final merge imo. Since they are quite
similar to the one in bigcouch (with small changes) maybe it would be
easier to just wait. @davisp @rnewson thoughts?

For the others features I will make them available in one commit until
tomorrow.

ToDo :

- add new features
- make rcouch use the splitted repositories created by @davisp (thanks a
  lot!) . It's actually blocked because of a missing change in
couch_mrview (unitests are missing). I will have a look on it later in
the day. Hopefully @davisp will be able to help me on that. For now my
attemps to resurrect them while preserving the history were a failure.


Voila,

Hopefully this merge will be finished by the end of the week. I profit
on this mail to announce that I will take a vacation of 15 days at the
end of the week, so I will be not available except for urgent things.


- benoit


[1] https://github.com/basho/lager
[2] https://github.com/refuge/rcouch/wiki/Validate-documents-on-read
[3] https://github.com/refuge/rcouch/wiki/Write-only-databases
[4] https://github.com/refuge/rcouch/wiki/Couch-changes#wiki-filter-fields-in-included-doc-when-returned-from-a-_changes
[5] https://github.com/refuge/rcouch/wiki/Source-Code-Changes#wiki-changes-in-core

Mime
View raw message