incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Cottlehuber <d...@muse.net.nz>
Subject Re: iterative map reduce
Date Wed, 14 Dec 2011 09:16:56 GMT
On 14 December 2011 10:05, Robert Newson <robert.newson@gmail.com> wrote:
> Chained map-reduce is not available in the open source BigCouch
> project, only on our hosted service at cloudant.com. Sorry!
>
> B.

A cryin' shame!!!

> On 14 December 2011 09:01, Alon Keren <alon.keren@gmail.com> wrote:
>> I've been working on an open-source tool with a similar approach chaining
>> map-reduce:
>> https://github.com/afters/Couch-Incarnate
>>
>> The need to pay the bills withheld me from putting more time into it
>> lately, so I can't say it's production quality.
>> Hopefully, this will change soon.
>>
>> Dominic - the method I currently use in production is to poll the changes
>> on my DB every few seconds and update a 'summary' DB accordingly. It's not
>> pretty, but it works. I hope to replace it with Couch-Incarnate asap.
>>
>>  Alon
>>
>> On 14 December 2011 10:50, Dave Cottlehuber <dave@muse.net.nz> wrote:
>>
>>> On 14 December 2011 05:46, Dominic Tarr <dominic.tarr@gmail.com> wrote:
>>> > hi,
>>> >
>>> > I need a iterative map reduce, prefurably, in couchdb.
>>> >
>>> > I want to be able to do a map reduce to generate unique items, then
>>> another
>>> > map reduce to generate stats about those unique items.
>>> >
>>> > from what I know about couchdb at the moment, it seems like id have to do
>>> > the first step at the document creation step, which is just ugly.
>>> >https://mail.google.com/mail/?shva=1#inbox
>>> > has anyone else dealt with this problem?
>>>
>>> Hi Dom,
>>>
>>> Cloudant do this in BigCouch which they released as open source a
>>> while back http://support.cloudant.com/kb/views/chained-mapreduce-views
>>> there are a few other posts on chained MR around but I don't know of a
>>> better Couch-style implementation. Their basic approach is to copy the
>>> view output into a new db which you'd then re-MR as required. Rinse
>>> and repeat until complete.
>>>
>>> Depending on your data it might be possible to implement their
>>> view->copy to db -> new view approach yourself in native couch. It
>>> would be interesting to do this as a couch extension using that nifty
>>> plugin interface that hasn't been developed yet!
>>>
>>> A+
>>> Dave
>>>

Mime
View raw message