Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of oil.crayons@gmail.com
 designates 209.85.212.200 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=JzteEqwmNI8m81gFiuky4PNS7JAf8v4zLsv6lKscRgnMw8tFJZuqhD4Bcg+yTTAc3D
         EZ6D4y5JFAJ5PgAJv3HwWJI6dfg8xAOnMwh58wbsRNnyOHTuvDoX+wjRjl4oAv+vqyJD
         CswDSePA/qN7NvjtIFe+Fq9qyEUByT5bSz5PE=
MIME-Version: 1.0
In-Reply-To: <20090707195222.GF7036@delora.autosys.us>
References: <261cf6280907041526n1d798770l16a0f4ec9f74e702@mail.gmail.com>
	<20090706085321.GA11394@uk.tiscali.com>
 <261cf6280907071229v4ab5d3feo992acd1a700214c7@mail.gmail.com>
	<20090707195222.GF7036@delora.autosys.us>
From: Dmitry Unkovsky <oil.crayons@gmail.com>
Date: Thu, 9 Jul 2009 12:12:22 +0300
Message-ID: <558050c20907090212g3e66e543wf2d3590be5d215d7@mail.gmail.com>
Subject: Re: View Performance (was Re: The 1.0 Thread)
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Can please someone give a very brief outline, how do I build erlview?
Or just a  pointer to somewhere on internets to read for it.

I'm no expert in erlang (well, yet), and running make in src gives
some arity error that i'm not capable to deal with for now.
This refers to checked out from git://github.com/mmcdanie/erlview.git
master branch (that is at 3e49de6998... "added cheesy helper
wrapper...")

Sorry for disturbing, probably I'd better go reading more docs, but as
they say "all work and no play makes Jack a dull boy", if it's
question of a few lines then I'll ask and you please forgive noob ;)

Best, Dmitry.

2009/7/7 Michael McDaniel <coucdby@autosys.us>:
> On Tue, Jul 07, 2009 at 12:29:08PM -0700, Scott Shumaker wrote:
>> The erlang view server does sound like it would be a big performance
>> win - although I think it will make CouchDB meet with more resistance
>> to mass-adoption if it becomes the preferred view path - simply
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> =A0Though someone could write a parse transformer so that
> =A0scripts entered via, e.g. Futon, could be closer to the
> =A0default javascript syntax, even when using an Erlang
> =A0view server (e.g. erlview).
>
> =A0Or even write wrappers which used more generally-familiar
> =A0syntax. =A0I started to do that with erlview, write wrappers
> =A0which helped simplify view writing.
>
> ~Michael
>
>
>> because Erlang is a much less common language than, say, Javascript -
>> and Javascript is a more natural fit to deal with JSON data. =A0Right
>> now, I don't have to train my team up on a new language to merely be
>> clients of CouchDB - and Javascript is far less intimidating and well
>> documented to the novice. =A0For us, there's additional benefits since
>> most of our codebase is in Javascript anyway - so our views can
>> leverage functionality by including code from our app. =A0But if the
>> erlang view server performance turns out to be significantly better,
>> we'll probably have to bite the bullet and rewrite all of our views.
>>
>> Still, I'm pretty convinced that you could make optimizations to make
>> the javascript view server nearly as fast as the erlang one -
>> especially since, when you really think about it - data comes into
>> CouchDB as JSON, gets run through a Javascript view server, and gets
>> returned to the client as JSON. =A0With JSON in and out (which is barely
>> touched by CouchDB - other than _id and _rev), it does seem like you
>> could avoid most of the conversion overhead. =A0And you could certainly
>> re-architect couchjs to have a more efficient communication path.
>>
>> I know the standard response is 'you want it - well, patches are
>> welcome!' but I'm speaking as a client of CouchDB - one that is very
>> busy trying to get a product launched (and not blessed with enough
>> free time at the present moment to really learn Erlang, let alone
>> contribute significant amounts of code). =A0So I'm just trying to be an
>> advocate for the many developers who would love to use (or are using)
>> CouchDB as a 'better SQL' - a persistent storage backend that makes a
>> lot more sense for storing and dealing with the kinds of data most web
>> apps face. =A0I think there are lot more people like me out there, too -
>> and tons more who are waiting with bated breath to see how CouchDB
>> develops.
>>
>> On Mon, Jul 6, 2009 at 1:53 AM, Brian Candler<B.Candler@pobox.com> wrote=
:
>> > On Sat, Jul 04, 2009 at 03:26:23PM -0700, Scott Shumaker wrote:
>> >> 53K docs apparently take 68s to be converted to JSON, and received by
>> >> the dummy server (with no docs emitted) - or about 780 docs/second.
>> >> couchjs is slower than bork.rb in this case (unsurprising - =A0bork.r=
b
>> >> not really parsing the data)
>> >> filtering on the couch side is an enormous win for our test case.
>> >>
>> >> K/V inserts - (5*53K in (200-68)s) =3D ~2000 per second
>> >>
>> >> This is a pretty big difference from Brian's results (8000/sec),
>> >> although we're dealing with many more docs, and without comparing
>> >> hardware specs, it's difficult to draw conclusions.
>> >
>> > Hmm, my hardware is pretty lame (Thinkpad X30 laptop, P3-M 1.2GHz, 1GB=
 RAM,
>> > 2.5" IDE drive). Maybe it's because you're using a substantially large=
r
>> > dataset than mine, e.g. more Btree nodes to update or more flushes to =
disk
>> > or something. I'll try increasing the size of my benchmark test, but
>> > annoyingly RestClient inherits Net::HTTP's 60-second request timeout, =
and
>> > the high-level RestClient.get / RestClient.put API doesn't allow it to=
 be
>> > overridden. I'll probably monkey-patch Net::HTTP for simplicity.
>> >
>> > Regards,
>> >
>> > Brian.
>> >
>
> --
> Michael McDaniel
> Portland, Oregon, USA
> http://autosys.us
>
>