couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Smith <jason.h.sm...@gmail.com>
Subject Re: [PROPOSAL] Allow rewrites to be JS function
Date Mon, 28 Sep 2015 08:22:03 GMT
On Mon, Sep 28, 2015 at 5:11 AM, ermouth <ermouth@gmail.com> wrote:

> I‘ve taken that PR, cleaned it up, modified for chttpd and fixed several
> most hitting performance issues. Right now performance isn‘t so terrible.
>

At first glance, my guess about performance is that there are two problems
(both stemming from the unfortunate architecture that JS is a separate OS
process, connected to the core via synchronous stdio messages).

1. In a naive implementation at least, every request needs to be forwarded
to a JavaScript process over stdio, and a response needs to come back. That
JS process is unavailable for other things (like map/reduce indexing) until
it responds. In other words, lots of serialization and inter-process
messages

2. If the JS engine is not running yet (or if the design doc has changed,
or if its crashed, etc.) then the core needs to exec a new child process.
So now you are talking about a CGI-like architecture.

If those are the issues, then I'd say it's more of a "scalability" problem
than a "performance" problem. Some couches have 50,000 or 100,000
databases. For example, one database per user--and I've even seen worse
situations where every *pair* of users might have a shared database, N
users choose 2 -> a lot.

So that might be a situation where people may get concerned about
performance: wanting to avoid spinning up a custom JS process for
potentially every unique user.

The last time I thought about this (years ago), I decided, first off, I do
not like the custom rewrite language. It works, but it I would describe as
"some custom, idiosyncratic system somebody thought up one day."

I've thought about a series of more modest partial steps along the way:
regular expression rewrites. I thought it might roll out in two steps:

1. Basically swap the current system with Perl-compatible URL rewrites. So
if you know Perl or sed, now you can rewrite couch requests.
2. Expand on step 1 by allowing matches against not only the requested URL,
but also the HTTP method and any request headers.

One advantage is that rewrites remain declarative, the "hard worK" still
done in the Erlang HTTP code. So it would be hard to argue performance or
scalability problems without also admitting that the current system is
already bad. (In my opinion, the correct position is, yes, yes it is in
fact already bad, but hey we're all still alive.)

And speaking politically, asking the developers to replace an old, custom
system with something easy to understand, from Erlang core--that is an
easier sell.

>From experience, I can say, one disadvantage to this approach is that
people haul out that shallow, pretentious JWZ quote as a substitute for
clear, careful thinking, and before you know if, you've got a bike shed
argument on your hands.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message