incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <rnew...@apache.org>
Subject Re: Options for Iterative Map Reduce
Date Wed, 12 Dec 2012 22:14:43 GMT
Yes, our chained map-reduce is incremental.

On 12 December 2012 22:07, nicholas a. evans <nick@ekenosen.net> wrote:
> On Wed, Dec 12, 2012 at 4:03 PM, James Marca
> <jmarca@translab.its.uci.edu> wrote:
>> I feel your pain but cannot offer any help.  I also use your option 5:
>> I use node.js to manually store view output into a separate db, with
>> the doc _ids equal to the key of the view output, so that I can limit
>> updates to only those things that change.
>
> Thanks James.  Do you apply the changes incrementally, and if so how
> do you detect which view rows (in the source DB) have changes so you
> don't need to download the whole reduced/grouped view?  And to the
> point of my last email, how do you detect missing view rows in the
> source DB and delete them from the chained DB?
>
>> This is one feature I really want in CouchDB...the ability to reduce
>> view output and/or treat view output like a regular couchdb database.
>>
>> Like a flag or something that says, hey couchdb, save this view as a
>> db automatically, but also update it when the source db changes (not
>> on view, but on edit)
>
> Agreed.  As far as I can tell, that's basically the Cloudant "dbcopy"
> feature.  I'm assuming their dbcopy updates incrementally.  On the
> other hand, the most common use case is probably alternate sort
> functions.  In that case, copying the view to a new DB seems wasteful
> (of time, disk space, and programmer time), when sort_by functions
> which generate secondary indexes should do the trick (and would be
> easily kept in sync with the view they are indexing).
>
> I wish I knew enough Erlang to dig in and see if either secondary
> indexes or a view changes feed is nasty complicated or trivial.
>
> Hey, I spent the last two weeks learning Lua in order to do some
> complicated Redis scripting.  How hard can Erlang and CouchDB
> internals be?  ;-)
>
> --
> Nick

Mime
View raw message