Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of jchris@gmail.com designates
 209.85.220.211 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:date:x-google-sender-auth:message-id:subject
         :from:to:content-type:content-transfer-encoding;
        b=xW+CYg8vOfLTOr8rEilZLrt+jgFH4xngdE2Rvjd/AQOX1y8gNBa7P1skTSu1JYQhkC
         /jl76aKW+pUrZgkbxupneTe14c/lLH1Lm8BMg140R++r0f+oZSoEsnYbOlC+SKu+saSv
         agjmvPrpLZR2rd7p6HIPd4Z4FuFkRPx/fVRjw=
MIME-Version: 1.0
Sender: jchris@gmail.com
Date: Thu, 4 Jun 2009 22:47:21 -0700
Message-ID: <e282921e0906042247j6bc0e2f3na69cf1516500dc92@mail.gmail.com>
Subject: chaining map reduce in hovercraft
From: Chris Anderson <jchris@apache.org>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

I finally got around to writing my map reduce copier. it's still
basic, but what do you think?

I want to put it into trunk as an http call, like:

POST /_snapshot_view

with JSON

{"src":"/srcdb/_design/app/_view/reduce_count", "group_level":2,
"target":"/targetdb"}

Chainable map reduce seems to be one of the most popular requests on
the survey we took, so hopefully this will make the heavy-data crew
happy.

There is an implementation here:

http://github.com/jchris/hovercraft/commit/34b44527b660a740858cc71aa2c8326747465e31#L0R290

What this does is take the results you'd get from query your reduce
view with group=true, and copy them to a new database. Basically you
end up with a database full of docs that look like:

{
"key":[2009,2,14],
"value": 511
}

Since they are docs sitting in another CouchDB, you can use more
ordinary CouchDB Map Reduce views on that database to do things like
sort by value, so you can for instance sort tags by popularity, or
days by user activity, etc.

Chris


-- 
Chris Anderson
http://jchrisa.net
http://couch.io