Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of jchris@gmail.com designates
 209.85.212.177 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=NibetMb32bWJ2nyPAGn9Jt2Fgz2j5phpKr2R+YXmjuu8UVTNRHTUdL/ov1RwgVDD4g
         hS5pOyxCRU6TDq73kJpjaWIpZpT9E6tl8DTwPDYKAixci1A6pdIn0cIPTL3LeWry28Pq
         wInusvhWaMUNjM34tlsSZoz0TAa+1jLD7yV90=
MIME-Version: 1.0
Sender: jchris@gmail.com
In-Reply-To: <fbbaf7b0908100801h61a8fe8dm4618d4f1ed3ee60e@mail.gmail.com>
References: <fbbaf7b0908100801h61a8fe8dm4618d4f1ed3ee60e@mail.gmail.com>
Date: Mon, 10 Aug 2009 12:30:08 -0700
Message-ID: <e282921e0908101230ie38d742x531e69a73295c4bc@mail.gmail.com>
Subject: Re: pmap and document update example
From: Chris Anderson <jchris@apache.org>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Mon, Aug 10, 2009 at 8:01 AM, Norman Barker<norman.barker@gmail.com> wro=
te:
> Hi,
>
> I commented on this in a recent thread, a couple of minutes later I
> had a working example (compliments to the modular design of couchdb
> and the easy set up with an ini file).
>
> Folloing Joe Armstrong's book on pmap I have
>
> update_db(JsonObject, Db)->
> =A0 =A0NewDoc =3D couch_doc:from_json_obj(JsonObject),
> =A0 =A0DocId =3D couch_util:new_uuid(),
> =A0 =A0{ok, _NewRev} =3D couch_db:update_doc(Db, NewDoc#doc{id=3DDocId}, =
[]).
>

You should just change this function to accept a list of docs.
couch_db:update_docs takes a list.

Writing a batch of docs in serial is much faster than doing it in
parallel would be (if you could even get it to work).

>
> pmap(F, L, Db) ->
> =A0 =A0S =3D self(),
> =A0 =A0%% make_ref() returns a unique reference
> =A0 =A0%% we'll match on this later
> =A0 =A0Ref =3D erlang:make_ref(),
> =A0 =A0Pids =3D map(fun(I) ->
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 spawn(fun() -> do_f(S, Ref, F=
, I, Db) end)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 end, L),
> =A0 =A0%% gather the results
> =A0 =A0gather(Pids, Ref).
>
> do_f(Parent, Ref, F, I, Db) ->
> =A0 =A0Parent ! {self(), Ref, (catch F(I, Db))}.
>
> gather([Pid|T], Ref) ->
> =A0 =A0receive
> =A0 =A0 =A0 =A0 =A0{Pid, Ref, Ret} -> [Ret|gather(T, Ref)]
> =A0 =A0end;
>
> gather([], _) ->
> =A0 =A0[].
>
>
> and then I invoke this with
>
> =A0 =A0{FormResp} =3D my_update_query_servers:render_update(Lang, UpFun,
> nil, nil, Req, Db),
> =A0 =A0case proplists:get_value(<<"results">>, FormResp, nil) of
> =A0 =A0 =A0 =A0undefined ->
> =A0 =A0 =A0 =A0 =A0 =A0throw({error, no_result});
> =A0 =A0 =A0 =A0ResultList ->
> =A0 =A0 =A0 =A0 =A0 =A0Res =3D pmap(fun update_db/2, ResultList, Db)
> =A0 =A0end,
>
>
> is there something bad with this? my_update_query_servers returns a
> collection of JSON objects and this seems to me to be a parallel
> write. =A0Since I gather all the references I should be able to update
> the caller with the list of created documents?
>
> Norman
>


--=20
Chris Anderson
http://jchrisa.net
http://couch.io