Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of mgenereu@gmail.com designates
 74.125.83.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=B1M+5B4gNNUV+X1S6MhgWS5F8eAlfAgTCNmqB9+IID1NkCIHE+b2EkuiujWO851FFY
         0m1Li5Yyc5GzumKT5ZuJepv+1USNte1MY7v6Cv3MmANxnCu4JbfiAoFSeMI3LOuy/JKA
         i7H1lkOtZwjEeszaGFws26bXitTeHZkM4Y9zQ=
MIME-Version: 1.0
In-Reply-To: <4EDCD199-A285-4510-B8F0-E6030BA9553D@apache.org>
References: <y2we2f5da511004181201sbba7d3ecy3c996dd8e59a9c85@mail.gmail.com>
	<4EDCD199-A285-4510-B8F0-E6030BA9553D@apache.org>
From: Michael Genereux <mgenereu@gmail.com>
Date: Mon, 19 Apr 2010 11:21:42 -0700
Message-ID: <v2ye2f5da511004191121sd69270efhd5784859e7a6f4ca@mail.gmail.com>
Subject: Re: Idea for View & Bulk Insert combination for ad-hoc database
	changes
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks!  Okay, I need to reread "_changes" and the "all_or_nothing" on
the bulk_api.  Also "Document Update Handlers" was way too complicated
and wanted a record id for what I was looking to do.  I have a CouchDB
database holding a log and I want to chuck old data easily.  Losing
quick manipulation when switching to CouchDB seems like a barrier of
entry.  I don't know of any situation where I haven't used SQL
(database), sed (ascii document), or other ad hoc concepts to
manipulate records/files/documents.

Thanks for the feedback and it was expedient so thanks for that too.

On Mon, Apr 19, 2010 at 9:45 AM, J Chris Anderson <jchris@apache.org> wrote=
:
>
> On Apr 18, 2010, at 12:01 PM, Michael Genereux wrote:
>
>> Reposting this from the user list. =C2=A0Figured it belongs in the dev l=
ist.
>>
>
> Sorry to leave you hanging for so long. (Vacation weekend!)
>
> There are 2 reasons CouchDB doesn't have this feature. The important one =
is that it could easily give new users the wrong idea. Coming from a RDBMS =
background you might expect something like this to be transactional. In Cou=
ch, it wouldn't be.
>
> Generally in Couch we've pushed work to the database client. When the cli=
ent is running on the same box as CouchDB, there would be 0 savings in proc=
essing overhead to run these transforms on the server, vs using a applicati=
on script.
>
> The second reason is the amount of code it would require. If someone want=
s to write it, there's a decent chance it'd be accepted as a patch (if it w=
ere done right.)
>
> Here's how I think it could be implemented (actually a more generic featu=
re that can be used for lots of things.)
>
> A design document could have a ddoc.changes function, which subscribes to=
 the changes feed of the database it runs in. The changes function could th=
en do anything it wanted, like send an email anytime a doc is saved that ha=
s a doc.send_me_as_email =3D true field.
>
> Your use case can be accomplished by having a function that watches to se=
e if any of the docs have a price that hasn't yet been updated, and change =
the price. So it could load the document, change the price, and set a flag =
on the document that says doc.price_changed =3D true. It would ignore any d=
ocuments that already had price_changed.
>
> This is almost exactly how you would accomplish this at the application l=
evel. The only reason to pull such an operation into the design document is=
 that CouchDB would take care of keeping it running, and that there would b=
e a standard way to author a changes listener.
>
> I'm not sure the status of it (node.js is a moving target) but Mikeal has=
 something much like this already on github:
>
> http://github.com/mikeal/node.couch.js/
>
> I'd suggest understanding how this works (and why it's like it is) before=
 thinking about extending CouchDB. The non-transactional nature of CouchDB =
means you need to understand the _changes feed before you can think about d=
oing anything "complete" like updating all docs that match a given pattern.
>
> Chris
>
>
>> I was doing some ad-hoc UPDATEs and DELETEs on a SQL database the
>> other day and it crossed my mind, how could I do the same on CouchDB?
>> I don't want to write an application to do something so simple. =C2=A0It
>> seems to me that I should be able to produce a Javascript views, both
>> named and temporary, that in turn produce results that get imported
>> into the CouchDB Bulk Document API as a single transaction.
>>
>> Here's the update example for a 10% increase in prices:
>> function( doc ) {
>> doc.price =3D doc.price * 1.1;
>> emit( null, doc )
>> }
>>
>> Here's the delete example for old records:
>> function( doc ) {
>> if( doc.year =3D=3D 2009 ) {
>> =C2=A0doc._deleted =3D true;
>> =C2=A0emit( null, doc )
>> }
>> }
>>
>> Also, much like INSERT ... SELECT notation, this could be used to copy
>> records. =C2=A0No need for the non-http compliant COPY method that does =
the
>> same on a single record level. =C2=A0Changes can be made to the copy on =
the
>> fly. =C2=A0Very efficient since the duplication and the update occur in =
one
>> transaction.
>>
>> Example to duplicate all 'foo' widgets to a set of 'bar' widgets:
>> function( doc ) {
>> if( doc.widget_type =3D=3D 'foo' ) {
>> =C2=A0reset( doc ); =C2=A0// helper function that does:
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 // delete( doc._=
id ); delete( doc._rev );
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 // and any other=
 special vars in future
>> =C2=A0doc.widget_type =3D 'bar'
>> =C2=A0emit( null, doc )
>> }
>> }
>>
>> I like not having to learn another command, so I reused emit even
>> though bulk api won't use the first field. =C2=A0If this feature existed=
,
>> this is how I would have expected to use it knowing that the key
>> parameter has no value in this use of a view.
>>
>> I would love some feedback on this.
>>
>> Michael
>
>