Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 68452 invoked from network); 5 Jun 2009 14:14:01 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 5 Jun 2009 14:14:01 -0000 Received: (qmail 70675 invoked by uid 500); 5 Jun 2009 14:14:12 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 70596 invoked by uid 500); 5 Jun 2009 14:14:12 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 70586 invoked by uid 99); 5 Jun 2009 14:14:12 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jun 2009 14:14:12 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of zachary.zolton@gmail.com designates 209.85.218.218 as permitted sender) Received: from [209.85.218.218] (HELO mail-bw0-f218.google.com) (209.85.218.218) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jun 2009 14:14:01 +0000 Received: by bwz18 with SMTP id 18so1780185bwz.11 for ; Fri, 05 Jun 2009 07:13:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type :content-transfer-encoding; bh=YHuwcxBIS/+08BOfIm2EwSXEJykEu/i7Ps4mzaxTqJ8=; b=euQXJs7Q4WIAbj9soyQhrV90xs7Cn2tohYnuu+ilogQQbprIzOq6l9jOIob+MqGjRk zoApNsuml+QzAzh/uM9cYEVBW3tII9SERII+Fq2CZaqHG+seri/DPFaEHIx1kKexd54G R3J52fU/ihh+JI6F26xYM4j43tkHVO+nNKHlw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=bgf+ByyVlkDj9ZXjaICz2GQGtT/zqqs824SoKQg4jB3nLH22hTdQ9OP2qHaZL7c7NK tQeVVFtTyPeCVXy3FwByWzR2UGv1VPFxzNmf2eOTvBxRqhOJOpE7+PVzQC/zPa8tB5Lh g7kOz81zzNLG3PkT2tvtCn+XrrRPB9NskkyEE= MIME-Version: 1.0 Received: by 10.216.47.213 with SMTP id t63mr1219917web.134.1244211219157; Fri, 05 Jun 2009 07:13:39 -0700 (PDT) In-Reply-To: <4A28FEC3.9050003@avicomp.com> References: <4A28FEC3.9050003@avicomp.com> From: Zachary Zolton Date: Fri, 5 Jun 2009 09:13:19 -0500 Message-ID: Subject: Re: chaining map reduce in hovercraft To: dev@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org So, Chris, it sounds like you're saying that POSTing to that URL will place the entire results of querying the view with group=true into another database. Sounds great! Will it work with 0.9? Would you suggest automating this using _changes? Cheers, Zach On Fri, Jun 5, 2009 at 6:17 AM, Viacheslav Seledkin wrote: > Chris Anderson wrote: >> >> I finally got around to writing my map reduce copier. it's still >> basic, but what do you think? >> >> I want to put it into trunk as an http call, like: >> >> POST /_snapshot_view >> >> with JSON >> >> {"src":"/srcdb/_design/app/_view/reduce_count", "group_level":2, >> "target":"/targetdb"} >> >> Chainable map reduce seems to be one of the most popular requests on >> the survey we took, so hopefully this will make the heavy-data crew >> happy. >> >> There is an implementation here: >> >> >> http://github.com/jchris/hovercraft/commit/34b44527b660a740858cc71aa2c8326747465e31#L0R290 >> >> What this does is take the results you'd get from query your reduce >> view with group=true, and copy them to a new database. Basically you >> end up with a database full of docs that look like: >> >> { >> "key":[2009,2,14], >> "value": 511 >> } >> >> Since they are docs sitting in another CouchDB, you can use more >> ordinary CouchDB Map Reduce views on that database to do things like >> sort by value, so you can for instance sort tags by popularity, or >> days by user activity, etc. >> >> Chris >> >> >> -- >> Chris Anderson >> http://jchrisa.net >> http://couch.io >> >> . >> >> > > The process of updating of shapshot db will be incremental? >