couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Chesneau <>
Subject Re: Input validation and limits
Date Tue, 26 Mar 2013 11:10:03 GMT
On Mon, Mar 25, 2013 at 11:45 AM, Jason Smith <> wrote:
> Yes this definitely informs the conversation about CouchDB as a data server
> / app server.
> In that discussion, I agree with you.
> CouchDB is susceptible to DOS attacks. For example, you can anonymously
> post to /_users until the disk fills up. (Maybe not in all deployments but
> in enough.) I have wanted a rate-limiting or CAPTCHA feature since I first
> learned CouchDB.

Yup totally agree, Imo having a request thresold or such things should
be builtin. Still not sure about the captcha, since it breaks
automatisation but why not in certain cases.

> No doubt there are many other features Couch needs to be competitive (the
> various authentication systems, for example). But, paradoxically, these
> examples are arguments *against* adding app server features, not for adding
> them. There is a limitless (or at least huge) list of features Couch would
> need to be a great app server platform. If each feature was hacked in, with
> its own /_config and /_whatever URL, then it could become a mess, for users
> and maintainers.

I also agree we should revisit the couchapp engine and somehow
separate it from the view engine which can be simpler. Also the HTTP
layer should be handled differently , as a fll transport and not mixed
with the db level so you can separate teh security problem (I still
think that the base of the couchdb core should be a simple document
oriented db with hooks to help the authz, and all the auth(z) done in
the transport). But that is another topic I guess.

- benoƮt

> On Mon, Mar 25, 2013 at 8:11 AM, Robert Newson <> wrote:
>> This is a great topic and one that goes to the heart of CouchDB's twin
>> roles as database and web server.
>> Does CouchDB need to directly support every feature that a web server
>> ought to support? Or does CouchDB, by virtue of speaking HTTP, get to
>> stay lean, providing only what must be provided by an origin server in
>> the modern Web, and rely on other, hopefully solid and focused tools,
>> for everything else? Supporting CAPTCHA, in whatever form, seems quite
>> reasonable. It's an extension of our auth model in many respects and
>> something that can't easily be externalized.
>> CouchDB's strength is that it's a database that speaks HTTP. In my
>> mind, it does that for one reason - to integrate with other things
>> that also speak HTTP. That obviously includes browsers but it also
>> includes load balancers, caching proxies, and so on.
>> To the topic at hand I feel that rate limiting and IP blocking is
>> something best done externally, just as I feel about virtual hosting
>> and URL rewriting. Are our log files rich enough to power fail2ban
>> itself? Could they be enhanced if not? Would an iptables approach to
>> rate limiting be preferable? Can we, as the CouchDB developer
>> community, really support and maintain all the extra features if we
>> decided CouchDB-as-a-web-server means it ought to do all these things?
>> Will we work to make a clustered CouchDB work without external load
>> balancers or DNS failover services, to pick just two examples? Will we
>> add an http caching layer?
>> I sound opinionated and entrenched when I ask too many questions in a
>> row, but they are sincere questions; it's not my intention to bludgeon
>> the proposal into the ground with them. I do want to explicitly reject
>> an accusation of "stop energy" before it's made, though. That phrase
>> is easily invoked though I do see that it's often been true in the
>> past, from myself and other developers.
>> Adding this kind of statefulness seems inappropriate to me but it's
>> hard to argue the case when we have the URL rewriting and virtual
>> hosting built in. A separate conversation is looming about virtual
>> hosting because the Nebraska merge that brings clustering will not
>> bring virtual hosting with it; BigCouch has never supported native
>> virtual hosting, it's provided by HAProxy instead.
>> I would love a broader discussion about where CouchDB ends and other
>> software begins. Is there a crisp line? I'd argue there could be,
>> though it's not crisp today. For me, as I've said, CouchDB is a
>> database that you talk to over HTTP. I'm for keeping that as lean as
>> possible; that's a big enough task already.
>> B.
>> On 25 March 2013 05:01, Jason Smith <> wrote:
>> > On Mon, Mar 25, 2013 at 4:14 AM, Alexander Shorin <>
>> wrote:
>> >
>> >> Hi Jason!
>> >>
>> >> On Mon, Mar 25, 2013 at 7:22 AM, Jason Smith <> wrote:
>> >> > ## reCAPTCHA support
>> >> > ...
>> >> > ## Rate limiting
>> >>
>> >> Wouldn't these things break bulk updates and replications? Both of
>> >> them triggers vdu much and let them fail on half way just because they
>> >> hit update rate wouldn't be nice.
>> >>
>> >
>> > Good point. This is why I wanted to have the discussion.
>> >
>> > I think the feature should be disabled by default. So upgrading CouchDB
>> > would not change how updates validate.
>> >
>> > I did mention a whitelist in my ideas, however I am not sure how it would
>> > work. That is why I identified "clients" rather than "users" or "remote
>> ip
>> > addresses." Do you whitelist users or ip addresses or {user,ip} tuples?
>> Or
>> > something else? And I don't think we want to start leaking client
>> > information into validate_doc_update. (Note that the req object is not
>> > passed to it, I think intentionally.)
>> >
>> > Anyway, yes, I agree, a client which fails validation very often
>> >
>> > Perhaps instead of a config, there could be a new flag when validation
>> > fails. So legacy code could never trigger banning.
>> >
>> >     throw {"forbidden":"Not allowed", "fail":true}
>> >
>> > So we might ban on "fail" frequency, not just anything thrown.
>> >
>> >
>> >> P.S. Currently, these questions could be solved via nginx in front of
>> >> CouchDB + fail2ban. May be better to integrate with existed tools? For
>> >> example, providing auth.log with authentication successful and failure
>> >> attempts - fail2ban will be happy for this. Currently you have to live
>> >> with verbose logs (or configure per-module logging, thanks to Jan!)
>> >> which looks a bit overhead if you're interested only in auth problems.
>> >>
>> >
>> > I have not looked at fail2ban for a while. However I am encouraged by my
>> > javascript.log work. I would love to make a fail2ban-friendly auth.log
>> file.
>> >
>> > --
>> > Iris Couch
> --
> Iris Couch

View raw message