incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alon Keren <alon.ke...@gmail.com>
Subject Re: iterative map reduce
Date Wed, 14 Dec 2011 09:01:53 GMT
I've been working on an open-source tool with a similar approach chaining
map-reduce:
https://github.com/afters/Couch-Incarnate

The need to pay the bills withheld me from putting more time into it
lately, so I can't say it's production quality.
Hopefully, this will change soon.

Dominic - the method I currently use in production is to poll the changes
on my DB every few seconds and update a 'summary' DB accordingly. It's not
pretty, but it works. I hope to replace it with Couch-Incarnate asap.

  Alon

On 14 December 2011 10:50, Dave Cottlehuber <dave@muse.net.nz> wrote:

> On 14 December 2011 05:46, Dominic Tarr <dominic.tarr@gmail.com> wrote:
> > hi,
> >
> > I need a iterative map reduce, prefurably, in couchdb.
> >
> > I want to be able to do a map reduce to generate unique items, then
> another
> > map reduce to generate stats about those unique items.
> >
> > from what I know about couchdb at the moment, it seems like id have to do
> > the first step at the document creation step, which is just ugly.
> >https://mail.google.com/mail/?shva=1#inbox
> > has anyone else dealt with this problem?
>
> Hi Dom,
>
> Cloudant do this in BigCouch which they released as open source a
> while back http://support.cloudant.com/kb/views/chained-mapreduce-views
> there are a few other posts on chained MR around but I don't know of a
> better Couch-style implementation. Their basic approach is to copy the
> view output into a new db which you'd then re-MR as required. Rinse
> and repeat until complete.
>
> Depending on your data it might be possible to implement their
> view->copy to db -> new view approach yourself in native couch. It
> would be interesting to do this as a couch extension using that nifty
> plugin interface that hasn't been developed yet!
>
> A+
> Dave
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message