couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mathias Leppich <mlepp...@muhqu.de>
Subject Re: Erlang reduce functions
Date Thu, 31 May 2012 06:49:26 GMT
I have recently moved a few views to erlang. One thing I don't like with the current couchdb
is that you have to choose the language for a whole design document. I sure understand that
this is due to the fact that all views in one design doc are generated at the same time and
so it makes sense that there are not ore that one view server involved, etc.

But my problem is that if you want your views to be build via erlang, you also have to convert
list and show functions to erlang, because you can not use list/show functions from design
doc A (js) with views from design doc B, right?

- mathias

On May 28, 2012, at 15:37 , Jeremy Smith wrote:

> Thank you for you explanation, looking forward to moving a number of our views to Erlang
> 
> Jeremy
> 
> 
> ________________________________
> From: david martin <david.martin@lymegreen.co.uk>
> To: user@couchdb.apache.org 
> Sent: Monday, 28 May 2012, 13:50
> Subject: Re: Erlang reduce functions
> 
> On 28/05/12 10:54, Jeremy Smith wrote:
>> Thank you, that works perfectly.
>> 
>> I guess the only way to understand how this works is for me to understand more about
Erlang, if you have any pointers/tips for learning Erlang I would be very grateful (I come
from a Perl, C, Java background)
> 
> In my humble opinion, your map function shows you have a very good understanding of Erlang
for having only started yesterday.
> 
> Your problem was not fully understanding the Couchdb reduce function.
> From what I have gathered, it has two possible phases of operation, "normal" reduce and
"re-reduce" reduce.
> Whether it is in normal reduce or re-reduce reduce is indicated by the third parameter
being true or false.
> 
> I have been experimenting in the area of Erlang view functions and am pleased to say
that, so far,
> all the Erlang function modules that Couchdb uses are available in the Erlang View function.
> The function can be debugged using the built in Log("string") Function combined with
formatting with  statements like
> 
>   Log(io_lib:format("X = ~p Val = ~p", [ Key,Value]))
> 
> Compile and run time errors are (very) fully  reported in the couch.log
> 
> I am using an even  balance of six cores without any specific attempts to use explicit
parallel maps.
> 
> The use of parallel maps would however enable heavy processing  of things such as serious
crypto functions
> i.e. sha512  or complex Perl Compatible Regular Expressions (PCRE)
> from the re module to be run in parallel over different Key/Values for each Document
in the View.
> 
> Try that in Javascript!
> 
> Very long serial computations can cause the gen-server handling the view to
> time out to stop runaway processes in the View code.
> 
> http://blog.vmoroz.com/2011/01/erlang-pmap.html
> 
> 
> The problem of not seeming to allow recursive calls in the view function has to be
> addressed by the use of the Y-combinator
> 
> http://blog.vmoroz.com/2010/10/recursive-anonymous-function-in-erlang.html
> 
> thanks to Victor Moroz for erudite explanations, well worth taking the time to understand
(if that is possible!).
> 
> I believe that the true versatility and power of Couchdb can only be realised
> using Erlang Views and there is very little in the way of examples on the web.
> 
> If there are any real specific interest cases I might even post some examples somewhere.
> 
>> Many thanks
>> Jeremy
>> 
>> 
>> ________________________________
>>    From: david martin<david.martin@lymegreen.co.uk>
>> To: user@couchdb.apache.org
>> Sent: Monday, 28 May 2012, 9:40
>> Subject: Re: Erlang reduce functions
>> 
>> try this
>> fun(Keys,Values,ReReduce)->
>>        case ReReduce of
>>            true->lists:sum(Values);
>>            false->length(Values)
>>        end
>> end.
>> 
>> 
>> On 27/05/12 16:33, Jeremy Smith wrote:
>>> Hi,
>>> 
>>> I am new to erlang, just started today, because we were advised to move some
of our views to native erlang views, I am trying to write a reduce function:
>>> 
>>> The Values its using seem to be in the form:
>>> [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1]]
>>> 
>>> 
>>> so I have tried to do the following to get a count of all the 1's, but it only
gives me the count of the outermost lists (in the above case 3, instead of 60(ish))
>>> 
>>> My Reduce funtion:
>>> fun(Keys, Values, ReReduce) ->   lists:flatlength(Values) end.
>>> 
>>> 
>>> My Map function looks like this:
>>> fun({Doc}) ->
>>>         Emitter = fun(Doc) ->
>>>             Date           = proplists:get_value(<<"publish_date">>,
  Doc),
>>> 
>>>             Year = string:substr(erlang:bitstring_to_list(Date), 1, 4),
>>>             Emit(erlang:list_to_bitstring(Year), 1)
>>>         end,
>>> 
>>>         HasRequiredFields = fun(Doc) ->
>>>             case {proplists:is_defined(<<"a_number">>, Doc), proplists:is_defined(<<"publish">>,
Doc), proplists:is_defined(<<"publish_date">>, Doc)} of
>>>                 {true, true, true} ->
>>>                     Emitter(Doc);
>>> 
>>>                 _->
>>>                     false
>>>             end
>>>         end,
>>> 
>>>         HasRequiredFields(Doc)
>>> end.
>>> 
>>> Cheers
>>> Jeremy


Mime
View raw message