couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Noah Slater <>
Subject Re: Input validation and limits
Date Tue, 26 Mar 2013 10:41:06 GMT
Bob, I think you hit the nail on the head with this. I think this is a very
important conversation to be having, for BigCouch and beyond. And I get the
impression that simplification and focus are going to be big themes this

On 25 March 2013 08:11, Robert Newson <> wrote:

> This is a great topic and one that goes to the heart of CouchDB's twin
> roles as database and web server.
> Does CouchDB need to directly support every feature that a web server
> ought to support? Or does CouchDB, by virtue of speaking HTTP, get to
> stay lean, providing only what must be provided by an origin server in
> the modern Web, and rely on other, hopefully solid and focused tools,
> for everything else? Supporting CAPTCHA, in whatever form, seems quite
> reasonable. It's an extension of our auth model in many respects and
> something that can't easily be externalized.
> CouchDB's strength is that it's a database that speaks HTTP. In my
> mind, it does that for one reason - to integrate with other things
> that also speak HTTP. That obviously includes browsers but it also
> includes load balancers, caching proxies, and so on.
> To the topic at hand I feel that rate limiting and IP blocking is
> something best done externally, just as I feel about virtual hosting
> and URL rewriting. Are our log files rich enough to power fail2ban
> itself? Could they be enhanced if not? Would an iptables approach to
> rate limiting be preferable? Can we, as the CouchDB developer
> community, really support and maintain all the extra features if we
> decided CouchDB-as-a-web-server means it ought to do all these things?
> Will we work to make a clustered CouchDB work without external load
> balancers or DNS failover services, to pick just two examples? Will we
> add an http caching layer?
> I sound opinionated and entrenched when I ask too many questions in a
> row, but they are sincere questions; it's not my intention to bludgeon
> the proposal into the ground with them. I do want to explicitly reject
> an accusation of "stop energy" before it's made, though. That phrase
> is easily invoked though I do see that it's often been true in the
> past, from myself and other developers.
> Adding this kind of statefulness seems inappropriate to me but it's
> hard to argue the case when we have the URL rewriting and virtual
> hosting built in. A separate conversation is looming about virtual
> hosting because the Nebraska merge that brings clustering will not
> bring virtual hosting with it; BigCouch has never supported native
> virtual hosting, it's provided by HAProxy instead.
> I would love a broader discussion about where CouchDB ends and other
> software begins. Is there a crisp line? I'd argue there could be,
> though it's not crisp today. For me, as I've said, CouchDB is a
> database that you talk to over HTTP. I'm for keeping that as lean as
> possible; that's a big enough task already.
> B.
> On 25 March 2013 05:01, Jason Smith <> wrote:
> > On Mon, Mar 25, 2013 at 4:14 AM, Alexander Shorin <>
> wrote:
> >
> >> Hi Jason!
> >>
> >> On Mon, Mar 25, 2013 at 7:22 AM, Jason Smith <> wrote:
> >> > ## reCAPTCHA support
> >> > ...
> >> > ## Rate limiting
> >>
> >> Wouldn't these things break bulk updates and replications? Both of
> >> them triggers vdu much and let them fail on half way just because they
> >> hit update rate wouldn't be nice.
> >>
> >
> > Good point. This is why I wanted to have the discussion.
> >
> > I think the feature should be disabled by default. So upgrading CouchDB
> > would not change how updates validate.
> >
> > I did mention a whitelist in my ideas, however I am not sure how it would
> > work. That is why I identified "clients" rather than "users" or "remote
> ip
> > addresses." Do you whitelist users or ip addresses or {user,ip} tuples?
> Or
> > something else? And I don't think we want to start leaking client
> > information into validate_doc_update. (Note that the req object is not
> > passed to it, I think intentionally.)
> >
> > Anyway, yes, I agree, a client which fails validation very often
> >
> > Perhaps instead of a config, there could be a new flag when validation
> > fails. So legacy code could never trigger banning.
> >
> >     throw {"forbidden":"Not allowed", "fail":true}
> >
> > So we might ban on "fail" frequency, not just anything thrown.
> >
> >
> >> P.S. Currently, these questions could be solved via nginx in front of
> >> CouchDB + fail2ban. May be better to integrate with existed tools? For
> >> example, providing auth.log with authentication successful and failure
> >> attempts - fail2ban will be happy for this. Currently you have to live
> >> with verbose logs (or configure per-module logging, thanks to Jan!)
> >> which looks a bit overhead if you're interested only in auth problems.
> >>
> >
> > I have not looked at fail2ban for a while. However I am encouraged by my
> > javascript.log work. I would love to make a fail2ban-friendly auth.log
> file.
> >
> > --
> > Iris Couch


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message