couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: rcouch merge status
Date Wed, 12 Feb 2014 19:22:25 GMT
On Wed, Feb 12, 2014 at 7:11 AM, Benoit Chesneau <bchesneau@gmail.com> wrote:
> Hi all,
>
> Sorry for this late status. I have been busy with a cold and needed work
> on the code.
>
> Anyway here is a quick status of what  have been done since the last
> status:
>
> 1) The view changes has been merged. It includes some code from what
> have beem given last year to the Apache foundation + unitests and a
> small refactoring that was already in action in other places I manage.
>
> The view change feature include:
>
> - index all changes in an index by sequences and by key: it's possible
>   to get all the changes of an index from a sequence or all changes for
> a key or in a range since a a sequence. Removed key are also returned
> which is new compared to the current code in the rcouch core core. (this
> new feature can also be useful later for someone that would like to
> build a feature chaining map/reduces steps eventually).
> - api to listen on index changes. You can retrieve all changes at once,
>   listen until a change happen (~= longpoll), listen continuously on all
> changes. Unlike the current changes internal api, this api is designed
> so it can be reused by any Erlang plugin the erlang way.
> - api to retrieve changes of a db using a view. If the indexation of
>   changes is enabled in a design doc, it will be used in place of the
> current view filter. If the indexation is not enabled the old and non
> efficient view filter is used.
> - Add a facility to index the views in a continuous manner. While this
>   api is designed internally to be reused, it's only enabled for now ehn
> listening on _changes. or by enable it in the change listener described
> above. When enabled the view index is refreshed evey N secs (can be set
> in the config file) or after N updates in the database (can be set in
> the config file).
> - The replication has also been been modified so you can replicate using
>   a view index as a filte. (And yes you can filter the changes using a
> key which is can give you a feature similar to the channels in the
> couchbase sync gateway)
> - the views changes has now unitests.
>
> 2) couch_log has been improved and is now using lager [1]. With this change
> instead to use our own way to log file we are using lager which improve
> the logging and  will make couchdb more tolerant in the face of large or
> many log messages, it won't out of memory the node.
>
> Note: Lager can handle multiple backend but we are for now only handling
> the file and console backend in the ini file. Other backends can be
> configured using the app.config file. Also the code is slightly
> different from the current one in rcouch and imported from a private
> branch in use somewhere: it allows to update the log configuration not
> only in the app.config but also in the ini and handle the log level /
> module.
>
>
> What is missing from rcouch:
>
> - validate doc on read [2]
> -  Write only databases [3]
> - Filter fields in included doc when returned from a _changes [4]
> - optimisations [5]
>
> Optimisations may be wait the final merge imo. Since they are quite
> similar to the one in bigcouch (with small changes) maybe it would be
> easier to just wait. @davisp @rnewson thoughts?
>

I'd prefer to delay adding new things to anything until after we've
completed the merge. Any work that isn't towards finishing the merge
will be effectively doubled as we have to account for that during the
merge stuff.

That is to say, commits to master (specifically, Erlang changes),
1843-feature-bigcouch, or 1994-merge-rcouch should be strictly to
getting the merge finished. Obviously that's with the caveat that bug
fixes for the release process are pushed through.

> For the others features I will make them available in one commit until
> tomorrow.
>
> ToDo :
>
> - add new features

Is this just a general "get back to doing new work" or was there
something you mentioned that I'm not connecting this to?

> - make rcouch use the splitted repositories created by @davisp (thanks a
>   lot!) . It's actually blocked because of a missing change in
> couch_mrview (unitests are missing). I will have a look on it later in
> the day. Hopefully @davisp will be able to help me on that. For now my
> attemps to resurrect them while preserving the history were a failure.
>

Yep. Intended to do this last night but ran out of steam. I'll take a
look right now and get back to you with what I find.

>
> Voila,
>
> Hopefully this merge will be finished by the end of the week. I profit
> on this mail to announce that I will take a vacation of 15 days at the
> end of the week, so I will be not available except for urgent things.
>
>
> - benoit
>
>
> [1] https://github.com/basho/lager
> [2] https://github.com/refuge/rcouch/wiki/Validate-documents-on-read
> [3] https://github.com/refuge/rcouch/wiki/Write-only-databases
> [4] https://github.com/refuge/rcouch/wiki/Couch-changes#wiki-filter-fields-in-included-doc-when-returned-from-a-_changes
> [5] https://github.com/refuge/rcouch/wiki/Source-Code-Changes#wiki-changes-in-core

Mime
View raw message