couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipe Manana (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1398) improve view filtering in changes
Date Wed, 15 Feb 2012 13:39:02 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208465#comment-13208465
] 

Filipe Manana commented on COUCHDB-1398:
----------------------------------------

BenĂ´it,

for 1) - Sure, no probem.

for 2) - That's right. It will work when the ddoc signature changes - my comment there was
if continuing to send changes after that happens is ok or not for applications (since the
view definition might have changed radically) - it's probably ok for many applications. I
don't have strong opinion here. If no one objects it can remain like that imho.

Now for the the case the ddoc is deleted (and never recreated again), the client will hang
forever. The process spawned with spawn/1 will terminate with a reason different from 'normal'
(with a thow {not_found, deleted}) but the parent will never know that, because it's not linked
to it or monitoring it, so it will hang forever waiting for db_updated or view_updated events
(the later won't come anymore). Like for normal continuous changes feeds we stop if the database
is deleted, for _view changes we should stop if the ddoc is deleted (or its "views" attribute
is deleted in an update) - this case seems to make sense.

for 3) - I'm ok with that.

for 4) - Non manual tests are good to have :)

I've also noticed after last comment that send_view_changes/2 in couch_changes.erl (https://github.com/benoitc/couchdb/compare/master...couch_view_changes#L4R372)
is buffering the full_doc_infos of every document id found in the view when folding it. This
seems it can buffer millions (or more) of full_doc_info records, no? If it's really unbounded
(as it seems just by looking at the diff), than it's dangerous (even a few thousand full_doc_infos
can be too much, if the revision trees have a big depth and/or many branches).,
                
> improve view filtering in changes
> ---------------------------------
>
>                 Key: COUCHDB-1398
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1398
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: View Server Support
>    Affects Versions: 2.0, 1.3
>            Reporter: Benoit Chesneau
>              Labels: changes, view
>         Attachments: 0001-white-spaces.patch, 0002-initial-step-move-the-code-from-couch_httpd_db-to-co.patch,
0003-fix-indent.patch, 0004-This-wrapper-is-useless-somehow-split-the-code-in-a-.patch, 0005-add-view_updated-event.patch,
0006-immprove-view-filter.patch, 0007-useless-info.patch, 0008-whitespaces.patch, 0009-handle-native-filters-in-views.patch
>
>
> Improve the native view filter `_view` support by really using view index. This patches
add following features
> - small refactoring: create the couch_httpd_changes modules, to put all the changes http
support in its own module instead having it in couch_httpd_db. 
> - add the `view_updated` event when a view index is updated : {view_updated, {DbName,
IndexName}}
> - start the feed using results in the view index instead of all the db index
> - only react on view index changes.
> For now next changes are still get using the couch_db:changes_since function and passing
the map function to the results. It could be improved if we had a by_seq btree in the view
index too. Other way may be to skip a number of the documents already processed. Not sure
it would be faster. Thoughts ?
> The branch couch_view_changes  in my repo contains preliminary support:
> https://github.com/benoitc/couchdb/tree/couch_view_changes
> Diff:
> https://github.com/benoitc/couchdb/compare/master...couch_view_changes
> To use it, use the native filter named _view which take the parameter view=DesignName/Viewname
> eg:
>   http://server/db/_changes?feed=continuous&heartbeat=true&filter=_view&view=DesignName/SomeView
> It has also an interresting side effect: on each db updates the view index refresh is
triggered so view updates are triggered. Maybe we could introduce an optionnal parameter to
not trigger them though?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message