couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Young <byo...@bigbluehat.com>
Subject RE: [PROPOSAL] Allow rewrites to be JS function
Date Tue, 20 Oct 2015 15:14:43 GMT
> I very much think a single shot application server is a great idea (see firebase), however
I am very much of the opinion that it should be built on top of CouchDB, not into it.

Agree with Dale. The architecture (last I checked) for these extension points (_rewrite, etc)
are (or once were) pretty modular, small, and replaceable. There was also work done once upon
a time for a proper plugins system.

Could we leverage that (or those) and create a handful of "primed" CouchDB distributions?
At least two: one that's "just" a database, and one that's an HTTP application server.

It's pretty easy to imagine a thing called "Apache CouchApp" that actually is a proper Firebase-style
product that's a sub-project of Apache CouchDB, rolls in all these endpoints, improves on
them, and doesn't piss off the Apache CouchDB foundation code contributors by being under
foot. ;) It could even have it's own release cycles, etc.

As long as a "CouchDB foundation" project stays extensible (as they are now; and better!),
then I think we could create a space where both a foundational JSON database and an eventual
application server based on that foundation could both thrive.

Seem reasonable? Or a bridge too far?

Love,
BigBlueHat


-----Original Message-----
From: Dale Harvey [mailto:dale@arandomurl.com] 
Sent: Tuesday, October 20, 2015 10:05 AM
To: dev@couchdb.apache.org
Subject: Re: [PROPOSAL] Allow rewrites to be JS function

On the server side you can run what ever JavaScript you want, it is far less limited than
client side JavaScript. PouchDB also runs on the server and has the exact same issue in terms
of seperation of concerns.

PouchDB doesnt expect developers to do dom manipulation, templating, scheduling and other
applications features via API's it provides, neither will it ever provide a reverse proxy

Pouch and CouchDB can both better integrate with things that do provide reverse proxying and
other such features much better though. Here is an example of a simple server that provides
a proxy on /proxy, and admin party database on /db and a read only database on /public (it
could use either pouchdb or couchdb to provide the databases)

https://gist.github.com/daleharvey/bf4e9a94553b75ef2d39

Its far from ideal, but already simpler than attempting the same setup with CouchDB only and
far far more extendable. I didnt need to wait for CouchDB to implement any features or discuss
their implementation.

While CouchDB is your only application platform, there will never be enough functionality
in it to fulfill even the basic use cases and people will lean towards better solutions (see
mongo + meteor).

I very much think a single shot application server is a great idea (see firebase), however
I am very much of the opinion that it should be built on top of CouchDB, not into it.

On 20 October 2015 at 14:37, Johs. E <johs@b2w.com> wrote:

> Hi Dale
> I dont understand the “integrate with their use case as best as we 
> possibly can” when it comes to rewrite and reverse proxy.
> As far as I know there is nothing to integrate here.
> The joy for PouchDB to be a pure database is very understandable, 
> since it sits in this rich environment that allow you to do anything.
> At the server side CouchDB allows for a very limited portion of 
> javaScript processing, it would be good if it was less restrictive.
> I am not advocating arbitrary application platform functionality.
> Just take out some limitations for the javascript crowd out there to 
> love CouchDB
>
> Thanks and best regards,
> Johs
>
>
> > On 20 Oct 2015, at 11:10, Dale Harvey <dale@arandomurl.com> wrote:
> >
> > This discussion has gone round and round a couple of times in 
> > different forms so I will avoid repeating my previous points but 
> > from working on PouchDB, the focus on having 'PouchDB' be a database 
> > only is fairly liberating, by not trying to add fairly arbitrary 
> > "application platform" features into the core codebase we can focus 
> > on integrating with what does provide those application platform 
> > features much better.
> >
> > Users do not need to wait for reverse proxying or url rewriting to 
> > be an agreed, implemented and maintained feature, they can use the 
> > many that already exist and we will ensure that we integrate with 
> > their use case as best as we possibly can.
> >
> > On 19 October 2015 at 23:57, Harald Kisch <haraldkisch@gmail.com> wrote:
> >
> >> Hi Garren,
> >>
> >> look at MSSQL (CLR) or ORACLE (JAVA Forms) any database was trying 
> >> to support their users with markup languages like XML, HTML, etc. 
> >> for
> instance
> >> directly out of the database core (performance, simplicity, 
> >> scalability,..).
> >> Lotus Notes did also integrate JavaScript inside of their core (Do 
> >> you
> know
> >> which guy did take part of it?). This have different reasons, but 
> >> one of this reasons is to support users with dynamic mutable data 
> >> directly into their GUI in JSON format which in my opinion is the 
> >> fundamental part of CouchDB to be a database for the web.
> >> Improvements get lost if we look at others and try not to be different.
> In
> >> my opinion we should more think about replacing spidermonkey with 
> >> the google V8 engine and itegrate node completely into the CouchDB 
> >> core to consume npm-packages directly instead of using them in the 
> >> local
> filesystem
> >> outside of CouchDB, where unfortunatelly complexity rise up at scaling.
> >>
> >> --Harald
> >>
> >> On Mon, Oct 19, 2015 at 9:24 PM, Johs Ensby <johs@b2w.com> wrote:
> >>
> >>> Hi Garren,
> >>> thanks for the "not standing in the way", I hope for more 
> >>> volunteers to iron out some of CouchDB's old akward wrinkles.
> >>> I am all with you for simplification:) ermouth's rewrite function 
> >>> is a huge simplifier.
> >>>
> >>> Where I disagree with you is where you say "probably a sign that 
> >>> this
> >> idea
> >>> is not something worth pursuing".
> >>> Whenever you discover that you have a differentiator, it's always 
> >>> a
> good
> >>> idea to look closely before discaring it and blend in with the rest.
> >>> It's all about attracting the next million web developers.
> >>>
> >>> johs
> >>>
> >>>
> >>>
> >>>
> >>>> On 19. okt. 2015, at 20.08, Garren Smith <gs@redcometlabs.com>
wrote:
> >>>>
> >>>> I’m really struggling with these proposals. I love the enthusiasm

> >>>> of
> >>> everyone but I keep thinking we should rather simplify CouchDB.
> >>>> CouchDB is ultimately a database. One with excellent sync
> capabilities.
> >>> And combining that with libraries like PouchDB and Hoodie make it 
> >>> an amazing database to build applications with.
> >>>> Adding routers and reverse proxies just makes it feel like we 
> >>>> trying
> to
> >>> push CouchDB into being more than it needs to be.
> >>>>
> >>>> For example building Couchapp like functionality in Node.js is so
> >> simple
> >>> and way better. Languages like Go also do that really well. Far
> superior
> >>> than what we can do with a database.
> >>>> I would rather let the Node.js and Go web libraries serve content 
> >>>> and
> >>> let us focus on building a clustered replicating database. We will 
> >>> draw more people to this community if we can do that properly over 
> >>> creaky,
> >> slow
> >>> and limited web serving mashed on top of a database.
> >>>> If I look at other popular databases, I don’t see any of them 
> >>>> serving
> >>> web content which is probably a sign that this idea is not 
> >>> something
> >> worth
> >>> pursuing.
> >>>>
> >>>> However if there is a burning desire for this and developers 
> >>>> raising
> >>> their hands to code this functionality, I would not stand in your way.
> It
> >>> is great to see the varied use of CouchDB out in the wild.
> >>>>
> >>>> Cheers
> >>>> Garren
> >>>>
> >>>>
> >>>>> On 19 Oct 2015, at 4:47 PM, Johs. E <johs@b2w.com> wrote:
> >>>>>
> >>>>> Thanks Andy,
> >>>>> I will try and  get some use cases up on confluence.
> >>>>> As for whoever would pick up the work after ermouth,  I have of
> course
> >>> one big thing on the wish list that goes well with a new router
> >> solution..
> >>>>> reverse proxy
> >>>>> I remember asking about it when I first started to work w 
> >>>>> CouchDB and
> >>> there were some concerns regarding security.
> >>>>> Since then I think node.js has paved the way with content 
> >>>>> scraping
> and
> >>> all sorts of outgoing traffic.
> >>>>>
> >>>>> Has anyone work on a reverse proxy solution for Couch?
> >>>>>
> >>>>> johs
> >>>>>
> >>>>>
> >>>>>> On 18 Oct 2015, at 21:36, Andy Wenk <andywenk@apache.org>
wrote:
> >>>>>>
> >>>>>> Hey Johs,
> >>>>>>
> >>>>>> thanks a lot for this. I need some time to dig into it. We need

> >>>>>> a
> >>> place to
> >>>>>> write the user stories / use case down. So I suggest we find

> >>>>>> good
> >>> place at
> >>>>>> the cwiki. So I suggest to use
> >>>>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/COUCHDB/Enhancement+Propos
> als
> >>> .
> >>>>>> Do you have write access there? If not, please ping me.
> >>>>>>
> >>>>>> Great work!
> >>>>>>
> >>>>>> All the best
> >>>>>>
> >>>>>> Andy
> >>>>>>
> >>>>>> P.S.: Jan already mentioned the feature freeze. Please take
it 
> >>>>>> not
> >> as a
> >>>>>> demotivation but as the possibility to have a bit more time
to 
> >>>>>> work
> >> on
> >>> it.
> >>>>>>
> >>>>>> On 17 October 2015 at 17:32, Johs Ensby <johs@b2w.com>
wrote:
> >>>>>>
> >>>>>>> Andy,
> >>>>>>> I will make my first use case for function in _rewrite a
high 
> >>>>>>> level
> >>> one:
> >>>>>>> to create a standalone server that is an all-in-one DB server,
> >>> application
> >>>>>>> server, api server and web server.
> >>>>>>>
> >>>>>>> I have played with the build of CouchDB 2 with rewrite 
> >>>>>>> function implemented that  ermouth put up on the irish AWS

> >>>>>>> community AMI
> list
> >>> and
> >>>>>>> the new use cases are endless.
> >>>>>>> First, I find that there are a few things that people fail
to
> notice
> >>> about
> >>>>>>> ddocs.
> >>>>>>> you need a tool to build a ddoc, editing JSON is not a viable
> >> option.
> >>> The
> >>>>>>> Ddoc Lab of ermouth is in a class of its own. If you havent

> >>>>>>> tried
> it
> >>> yet,
> >>>>>>> do so from http://ddoc.me/ <http://ddoc.me/>. Installing
on 
> >>>>>>> your
> >> own
> >>>>>>> couch it is as easy as storing the application, all included

> >>>>>>> as one document in any database. Ddoc Lab is a component

> >>>>>>> oriented IDE with
> >>> syntax
> >>>>>>> checking, less preprosessor and other build tools that let
you 
> >>>>>>> keep
> >> a
> >>> well
> >>>>>>> organized ddoc as a source project (in one couchdb document)

> >>>>>>> and
> you
> >>>>>>> publich a ddoc to any target db.
> >>>>>>> with this tool you can organize your js modules and templates

> >>>>>>> etc
> >> and
> >>>>>>> basically...
> >>>>>>> set up the API of your application in a ddoc. You can switch
> between
> >>>>>>> databases and their ddoc functionality based on username,
role 
> >>>>>>> or geolocation and limit access to parts of the Couch API
as 
> >>>>>>> needed
> >>>>>>>
> >>>>>>> This is the method I would recommend to explore powerful

> >>>>>>> simplicity
> >>> with
> >>>>>>> function in rewrites
> >>>>>>> redirect port 80 directly to couch set up 2 vhosts, one
for 
> >>>>>>> public access pointing to
> youdb/_design/api
> >>> and
> >>>>>>> one for sysadm pointing to /
> >>>>>>> for admin use Fauxton and Ddoc Lab on the sysadm vhost you
are 
> >>>>>>> set to develop great systems, no big tool stack to learn,
> >> just
> >>>>>>> bring in whatever js modules you like, the template engine
you
> like,
> >>> the
> >>>>>>> router you like, the HTML5 stuff you like..
> >>>>>>> .. or just write some very compact js code in one place
where 
> >>>>>>> you
> >>> ealier
> >>>>>>> had to mess around with a whole stack of tools and systems
> >>>>>>>
> >>>>>>> below is the req object that the function takes
> >>>>>>>
> >>>>>>> Johs
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> The rewrite function has this syntax
> >>>>>>> function(req) {
> >>>>>>>     .. your code that will
> >>>>>>>     return
> >>>>>>>             path
> >>>>>>>             // optional
> >>>>>>>             headers
> >>>>>>>             method // you can change this on the fly
> >>>>>>>             code
> >>>>>>>             body
> >>>>>>> }
> >>>>>>>
> >>>>>>> the function receives this req object method path raw_path

> >>>>>>> query headers
> >>>>>>>     Accept
> >>>>>>>     Accept-Encoding
> >>>>>>>     Connection
> >>>>>>>     Host
> >>>>>>>     Upgrade-Insecure-Requests
> >>>>>>>     User-Agent
> >>>>>>>     x-couchdb-vhost-path
> >>>>>>> body
> >>>>>>> peer
> >>>>>>> cookie
> >>>>>>> userCtx
> >>>>>>> db
> >>>>>>> name
> >>>>>>> roles
> >>>>>>> secObj
> >>>>>>>
> >>>>>>>> On 1. okt. 2015, at 13.40, Andy Wenk <andywenk@apache.org>
wrote:
> >>>>>>>>
> >>>>>>>> Johs,
> >>>>>>>>
> >>>>>>>> Yes for sure! That's always great. Maybe you can also
write 
> >>>>>>>> some
> >> user
> >>>>>>> stories (given when then) or scribble some graphics. 
> >>>>>>> Everything is
> >>> useful
> >>>>>>> and will fasten the process ;-)
> >>>>>>>>
> >>>>>>>> Cheers
> >>>>>>>>
> >>>>>>>> Andy
> >>>>>>>>
> >>>>>>>> On 1 Oct 2015 12:38, "Johs Ensby" <johs@b2w.com <mailto:
> >> johs@b2w.com
> >>>>>
> >>>>>>> wrote:
> >>>>>>>> Thanks for this Andy,
> >>>>>>>>
> >>>>>>>> I can contribute to the discussion of the feature seen
from a 
> >>>>>>>> user
> >>>>>>> perspective.
> >>>>>>>> Would it be appropriate to present some use cases?
> >>>>>>>>
> >>>>>>>> best
> >>>>>>>> Johs
> >>>>>>>>
> >>>>>>>>> On 1. okt. 2015, at 12.33, Andy Wenk <andywenk@apache.org
> >> <mailto:
> >>>>>>> andywenk@apache.org>> wrote:
> >>>>>>>>>
> >>>>>>>>> Johs,
> >>>>>>>>>
> >>>>>>>>> Let me please show the steps needed.
> >>>>>>>>>
> >>>>>>>>> * discuss the feature very clearly on the dev@.
Please make 
> >>>>>>>>> sure
> >>> that
> >>>>>>> core
> >>>>>>>>> developers as committers with commit bits are involved
> >>>>>>>>>
> >>>>>>>>> * code the feature. Make sure to implement tests
> >>>>>>>>>
> >>>>>>>>> * send a pull request and show it to dev@
> >>>>>>>>>
> >>>>>>>>> * finally the community will accept or decline the
feature 
> >>>>>>>>> (this
> >>> will
> >>>>>>>>> involve refactoring and changes)
> >>>>>>>>>
> >>>>>>>>> As Alex said. The PMC or Jan do not decide about
the feature.
> >>>>>>>>>
> >>>>>>>>> All the best
> >>>>>>>>>
> >>>>>>>>> Andy
> >>>>>>>>> On 1 Oct 2015 11:21, "Alexander Shorin" <kxepal@gmail.com
> >> <mailto:
> >>>>>>> kxepal@gmail.com>> wrote:
> >>>>>>>>>
> >>>>>>>>>> On Thu, Oct 1, 2015 at 12:07 PM, Johs Ensby
<johs@b2w.com
> >> <mailto:
> >>>>>>> johs@b2w.com>> wrote:
> >>>>>>>>>>> will you welcome ermouths rewrite contribution?
> >>>>>>>>>>
> >>>>>>>>>> The decision is depends on the implementation.
If it will 
> >>>>>>>>>> be
> >> good,
> >>> why
> >>>>>>>>>> not? Finally, CouchDB is open source project:
we cannot 
> >>>>>>>>>> forbid
> >>> people
> >>>>>>>>>> right for contributions, we only welcome them.
> >>>>>>>>>>
> >>>>>>>>>>> Arguments against couchapps has to do with
performance and 
> >>>>>>>>>>> the
> >>> folly
> >>>>>>> in
> >>>>>>>>>> competing with node.js.
> >>>>>>>>>>
> >>>>>>>>>> Performance question for the new _rewrite implementation
is 
> >>>>>>>>>> very depends on query server. Once it can process
this kind 
> >>>>>>>>>> of
> >>> functions,
> >>>>>>>>>> you may use something better than JS to gain
better performance.
> >>> That
> >>>>>>>>>> could be Erlang native query server, or luerl-based
one, or 
> >>>>>>>>>> else
> >>> you
> >>>>>>>>>> like.
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> ,,,^..^,,,
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Andy Wenk
> >>>>>> Hamburg - Germany
> >>>>>> RockIt!
> >>>>>>
> >>>>>> GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3

> >>>>>> 9588
> >>>>>>
> >>>>>> https://people.apache.org/keys/committer/andywenk.asc
> >>>>>
> >>>>
> >>>
> >>>
> >>
>
>
Mime
View raw message