couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <jch...@apache.org>
Subject Re: calulate doc popularity with couchdb map/reduce
Date Sun, 14 Jun 2009 18:04:45 GMT
On Sun, Jun 14, 2009 at 3:07 AM, Jan Lehnardt<jan@apache.org> wrote:
>
> On 11 Jun 2009, at 21:55, James Marca wrote:
>
>> On Wed, Jun 10, 2009 at 04:25:13PM -0500, Zachary Zolton wrote:
>>>
>>> So, another +1 for chainable map-reduce...? (-_^)
>>
>> And +1 from me too.
>>
>> I am completely ignorant of Erlang and the CouchDB internals, but why
>> is map/reduce output (all the view docs I see in the directory as
>> .whatever files) any different than regular CouchDB databases?  It
>> seems logical that the same tools that apply to a CouchDB database
>> should apply to view files as well, so one can map/reduce against view
>> without having to manually convert it back from view results into
>> another CouchDB database.
>
> The fundamental problem is that a reduced result can no longer keep
> track of which docs took part in creating this value. For incremental
> updates (knowing what to recompute when only a single doc changed)
> needs this.
>

I wrote some Erlang code to copy a reduce query to another database
for further sorting / viewing. I then realized it was not usable in a
multi-node cluster so I abandoned the idea of putting it in CouchDB.
Instead I plan to rewrite it in Ruby. It would seriously be about 6
lines of code so I'll see what I can do.



> Cheers
> Jan
> --
>
>>
>> Cheers,
>> James Marca
>>
>>
>>>
>>> On Wed, Jun 10, 2009 at 11:24 AM, Benoit Chesneau<bchesneau@gmail.com>
>>> wrote:
>>>>
>>>> 2009/6/10 Chris Anderson <jchris@apache.org>:
>>>>>
>>>>> On Wed, Jun 10, 2009 at 6:06 AM, Benoit Chesneau<bchesneau@gmail.com>
>>>>> wrote:
>>>>>
>>>>> It looks like you are building up a hash of scores in your reduce. Can
>>>>> you get the same results by keeping the reduce ignorant of keys, and
>>>>> just calculating the score for whatever keys are in the range. Then
>>>>> you can use a group query and sort by value on the client.
>>>>>
>>>>> Anything involving relative popularity of certain items will have a
>>>>> sorting phase outside of CouchDB's first pass of map reduce.
>>>>>
>>>>
>>>> hi, thanks for the feedback. I think you mean somethink like this :
>>>> http://www.friendpaste.com/4cIo2tzKA3epiwLkwPsolA
>>>>
>>>> It works yes,  but now I need a second phase to invert values/keys and
>>>> get them in right order. In fact having the possibility to save resukt
>>>> of first reduce in temporary docs and mapping them to a new result
>>>> could be a solution.
>>>>
>>>> - benoît
>>>>
>>
>>
>> --
>> This message has been scanned for viruses and
>> dangerous content by MailScanner, and is
>> believed to be clean.
>>
>>
>
>



-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Mime
View raw message