From user-return-16602-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Thu Jun 9 08:05:30 2011 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7092F6FB5 for ; Thu, 9 Jun 2011 08:05:30 +0000 (UTC) Received: (qmail 32931 invoked by uid 500); 9 Jun 2011 08:05:28 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 32892 invoked by uid 500); 9 Jun 2011 08:05:28 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 32884 invoked by uid 99); 9 Jun 2011 08:05:28 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jun 2011 08:05:28 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of cliffano@gmail.com designates 209.85.161.52 as permitted sender) Received: from [209.85.161.52] (HELO mail-fx0-f52.google.com) (209.85.161.52) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Jun 2011 08:05:24 +0000 Received: by fxm6 with SMTP id 6so1242557fxm.11 for ; Thu, 09 Jun 2011 01:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=5Og6+VFuwxwM9l55Er6ECm99te7SRLwQPi+JOy12GpE=; b=PzMRZHWBJrCjv/Mlprul9Zz28ZExtSdE4HD4CVjSO3eLOF0NP9CWV5sn/5Yt6IFpWV XLQeh6h67Oyg+43QbEUc4buE9LB7yM7uAE7kQCBkgKgXCLjJYnyJcZnqtzLV1keRHPL1 kVytqcgO7S1PnLmuGMYTFYXKuu3dj6nuvxNlQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=jHbTwMyGM+bGaw4hxmx+1VOPo78DFpo8z2OQ4+LStCv++VZzgQP0ABfnCjI5yRwJGR q3uv740xIya9k6odjxeGBhRIO4JZzE416I6Em3tKm28IlzQcRXtUCfJZZz4hT5TEY/cX WzZicbmTfs6S0ysXv+U32ZOCkBFO5ICT7pqQM= MIME-Version: 1.0 Received: by 10.223.62.194 with SMTP id y2mr417969fah.123.1307606702846; Thu, 09 Jun 2011 01:05:02 -0700 (PDT) Received: by 10.223.7.69 with HTTP; Thu, 9 Jun 2011 01:05:02 -0700 (PDT) In-Reply-To: References: Date: Thu, 9 Jun 2011 18:05:02 +1000 Message-ID: Subject: Re: [ann] couchtato - command line tool to apply 'offline' js functions against all docs in a database From: Cliffano Subagio To: user@couchdb.apache.org Content-Type: multipart/alternative; boundary=0015174769800775c504a542e9a9 --0015174769800775c504a542e9a9 Content-Type: text/plain; charset=ISO-8859-1 Temporary view, fully indexed, would perform faster of course. Remember that Couchtato works by iterating all documents in the database, so there's a cost there on top of network delay when retrieving those documents via http. However, if you want to find a document id where field foo === 'bar', then Couchtato is a good solution because you can just script a function that checks doc.foo === 'bar' and log the id, then fire off 'couchtato iterate -u http://your/db' Using Couchtato would require much less effort compared to adding/updating a view in a design document just to do a once-off search. Not to mention that modifying the design document might trigger view reindexing, etc. Another benefit of Couchtato is that one doesn't need to know about CouchDB view/design document in order to find that document. We have testers on our projects using Couchtato to find / count documents based on their test scenario, they don't need to modify CouchDB design document. Yea, it takes some time to get used to saying Couchtato :). On Thu, Jun 9, 2011 at 4:42 PM, Mark Hahn wrote: > Would it be faster to use couchtater to run a one-off search than using a > temporary view? > > P.S. Couchtato does not roll off the tongue. > > On Wed, Jun 8, 2011 at 11:32 PM, Cliffano Subagio >wrote: > > > Hi Mark, > > > > No, it doesn't read the db file directly. > > > > Couchtato interacts with CouchDB via couchdb's http api > > http://wiki.apache.org/couchdb/HTTP_Document_API using Cradle > > https://github.com/cloudhead/cradle . > > It reads the documents using linked list pagination as described here > > http://guide.couchdb.org/draft/recipes.html#fast , Couchtato then > applies > > all the functions described in its config file (on filesystem) on each > > document one by one. > > Writing / updating / deleting document are also done via http api. > > > > It can read about 230000 documents (CouchDB server on an external > network) > > in about 2.5 minutes, so that's about 1533 docs per second. > > > > The intention of this tool is to provide an easy way to 'do something' > with > > documents in couchdb without having to touch the design document on > couchdb > > itself. > > > > > > Cheers, > > Cliff. > > > > On Thu, Jun 9, 2011 at 3:06 PM, Mark Hahn wrote: > > > > > I assume it reads the db file directly. How does it do writes? > > > > > > On Wed, Jun 8, 2011 at 5:51 AM, Cliffano Subagio > > > wrote: > > > > > > > Hi all, > > > > > > > > I'd like to let you know about Couchtato - > > > > https://github.com/cliffano/couchtato > > > > It's a command line tool (using nodejs) that allows you to apply > > > javascript > > > > functions against all documents in a database, 'offline' in the sense > > > that > > > > those javascript functions live in a couchtato config file on your > > local > > > > filesystem instead of a view on CouchDB server. > > > > > > > > Couchtato is also handy for: > > > > - finding documents with a certain criteria > > > > - generating database dumps > > > > - updating, deleting, or counting documents > > > > > > > > Since Couchtato is just a simple command line tool, the users won't > > need > > > to > > > > know much about CouchDB / views / design documents, they only need to > > > know > > > > the database URL and to create some javascript functions - simple > > enough > > > > for > > > > non-developers. > > > > > > > > Hopefully this tool can be useful for someone out there. > > > > > > > > Cheers, > > > > Cliff > > > > > > > > > > > > > > > > -- > > > Mark Hahn > > > Website Manager > > > mark@boutiquing.com > > > 949-229-1012 > > > > > > > > > -- > Mark Hahn > Website Manager > mark@boutiquing.com > 949-229-1012 > --0015174769800775c504a542e9a9--