Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of nunojobpinto@gmail.com
 designates 66.249.92.168 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:references;
        b=KsFsw0UjUvE5Drs4w2Mf1r2By1L5GPoqKfLVqhtt/tZYVoVFrD//SWI8LJDFwEs8qX
         5glbO8IHmg1WbGw7PzYrbnKOdcA9sOA/I0ky31RGmsuPZLLXTtABA+ALavGDaIZVyXJA
         yLis8vPalLYnRdQ5gJSTtKzkZsjUil0nRDoDQ=
Message-ID: <30d0cf2c0811221256v4a4349f5w639c69a2c159f8e3@mail.gmail.com>
Date: Sat, 22 Nov 2008 15:56:53 -0500
From: "Nuno Job" <nunojobpinto@gmail.com>
To: couchdb-user@incubator.apache.org
Subject: Re: Map/Reduce takes lots of time every request
In-Reply-To: <e282921e0811221253r3af61404x7120458ddae09bb1@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_59977_9772583.1227387413524"
References: <49286ACA.4020606@gmail.com>
	 <e282921e0811221253r3af61404x7120458ddae09bb1@mail.gmail.com>

------=_Part_59977_9772583.1227387413524
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Slightly offtopic: Anyone saw Simon Peyton Jones talking about making a
generic mapreduce-like in haskell?  http://tinyurl.com/537apv

On Sat, Nov 22, 2008 at 3:53 PM, Chris Anderson <jchris@apache.org> wrote:

> On Sat, Nov 22, 2008 at 12:25 PM, maddiin <maddiin@googlemail.com> wrote:
> >
> > Do you have any advice what I am doing wrong and how I could speed this
> up?
>
>
> I'm curious how long it takes with reduce=false (should be limited
> basically by IO).
>
> I'm almost certain (please correct me if I'm wrong) that reduce
> requests must call the JavaScript interpreter at least once per
> request, to rereduce the btree inner-nodes that fit in that request
> range. This means for group=true requests, the rereduce function must
> run once per unique key (at minimum). That would be the source of your
> slowness. It sounds like you are building a tag-cloud. The smart money
> would be on caching the results of that operation, which is standard
> practice with SQL based tag clouds as well.
>
> If you're not doing a tag cloud, maybe there's a way you can get the
> needed results using map only?
>
> Also, I'm not sure, but perhaps it would be possible for CouchDB to
> cache final reduce values in the btree as well, so that group=true
> queries can save the cost of the final rereduce (and make subsequent
> queries fast...)
>
> Chris
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>

------=_Part_59977_9772583.1227387413524--