couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <>
Subject Re: CouchDB Module System
Date Mon, 23 Jan 2012 09:22:47 GMT
Add this one to the pile:

Improve config file write-back behavior.

On Mon, Jan 23, 2012 at 01:03, Randall Leeds <> wrote:
> On Sat, Dec 31, 2011 at 10:34, Jan Johannes <> wrote:
>> Dear favorite Document-database developer community :-)
>> This is my first message to the mailing list, so please forgive me and
>> gently correct me if i am breaking any communication standards whatsoeverŠ
> Nope! You're just bringing up a deep topic with a lot of text.
> I hope you didn't expect a speedy response! :)
>> I am currently working on a WebDAV interface for CouchDB, that now gets
>> closer to its release date, somewhere in February or March and as i
>> approach that, I want to package everything in a proper CouchDB Module
>> format but the CouchDB Module system is, as i understood, in early
>> development or pre-development phase and needs a bit of work.
>> So as i work on a CouchDB Module anyways i would like to contribute to the
>> Module system or start kicking the work on that of.
>> Hence the following questions:
>> Who started working on such a system or would be interested in talking
>> about the design decisions that need to be done before actual coding can
>> start?
>> (I know at least Jan Lehnardt expressed interest, but i can't quite
>> remember if he started actual work or just had plans for that.)
> Benoit is particularly interested and there is activity on the refuge
> project mailing list about it from Nicolas Dufour.
> I believe Jason Smith might be pretty interested since he's authored
> some "plugins" already.
> Others on the list are certainly interested.
> I'm keen to get a good module system and it's why I was in favor of
> the effort to get "couch-config" shipped as part of the install.
>> If there is any documentation on planning such a system, could someone
>> please point me to the recourses?
> There's an umbrella ticket I made for modularizing the code base more:
> And these:
> Utility to help plugin developers manage paths
> support load of external erlang modules in couchdb.
>> Lastly here is my two cents on a few topics that need to be addressed:
>> - We should build the best module system there is for a database or
>> content management system to gain back a bit of the traction we lost
>> during the recent plateau phase of couchdb enthusiasm
> I find it hard to judge enthusiasm from the inside, but, yeah... a
> proper module system would rock.
> One of the things CouchDB struggles with in terms of image, but excels
> at in terms of function is scaling up and down. The only way to really
> make one project move from mobile phone to datacenter cluster is to
> modularize the build and deployment of key components. I think
> everyone agrees about this. CouchDB is the Gentoo of document
> databases in this way.
>> - I think "module" is the preferable term, as extension, plugin or the
>> like sound like second class citizens, whereas module implies, that even a
>> core component of the system can be made into such an instance (Any other
>> suggestions?)
> Module is a little bit confusing because it conflicts with Erlang's
> definition. While the two would often be synonymous for many add-ons,
> it's not necessarily the case.
> I would go with calling them "applications" maybe. I've often thought
> that each such thing could correspond well to a single Erlang
> application and .app file, which Erlang has significant support and
> prior art for loading and unloading on the fly. But really, I don't
> care what color we paint this and long as it's neon awesome.
>> - A CouchDB module should be packaged as a stand alone couchdb-document,
>> with the yet to be specified manifest beeing the json structure of the
>> document and the files beeing the attachment.
> I'm pretty sure I disagree here. I think this conflates two issues.
> The first, to me, is a good hooks system from which more interactions
> in the core could be scripted with sandboxed user code. Once again, I
> know I've talked to Benoit about this. The second, though, is the more
> first-class type of application that requires compiled Erlang
> bytecode. While these have some degree of portability[1] I
> instinctively cringe, from a security standpoint, at the idea of hot
> loading code from a document into the VM. While the document-module
> idea I find attractive from a share-ability and install-ability
> standpoint, I don't think it befits the general purpose module system.
> Although, a good hooks system may make certain kinds of modules work
> this way.
> [1]
>> - Discovery, Installation, Enabling, Disabling, Module-Specific
>> Configuration and Distribution should be part of Futon and introduce as
>> little new interface concepts as possible, also the futon interface
>> section "configuration" may be needed to be integrated and streamlined
>> quite nicely
> Again, this works the dynamic-language, hooks-based module, but I'm
> not sure it works for all modules. Kanso [2] is doing this to a large
> extent already, under the CouchApp frame, but I see no reason why a
> hooks system couldn't accommodate a new, more powerful kind of Kanso
> package.
> [2]
>> - Even if a module is not possible to completely install via this method
>> (e.g. It depends on a os component, that is too low level etc.) such
>> modules should be as tightly integrated into the system as possible and
>> the manual admin installation steps should be documented in a way, so that
>> in a normal case no place but the module system needs to be accessed
> Agree.
>> - on the top level the system should be the same for all OSes
>> - The work packages i see at the moment for the structure i have in mind
>> would be:
>>        - Module repository: A CouchDB Instance, that hosts a collection of
>> CouchDB Modules
> See Kanso, above.
>>        - Futon User Interface to browse available Modules in the configured
>> repositories, should be close to the Firefox plugin paradigms
> That would be neat. Or a separate 'shopping' utility for CouchApps.
> Sounds a bit like a combination between JChris' CouchApp Garden [3]
> and Max Ogden's Chrome Couch application [4]. Crossing these would
> Kanso would be really neat.
> [3]
> [4]
>>        - Module Manifest: Descriptive CouchDoc, that references the attached
>> files or git repository, how files need to be fetched, packaged and what
>> goes where, also version                dependencies, description, rating,
>> Category, needed system privileges (basically if you need to be a couch
>> admin or if you need to be root on the os), needed              configuration,
>> basic description of the configuration parameters
> I'd really prefer to punt on low-level privileges and just call this
> two separate things. The way to unify them, in my mind, is to create a
> hooks system and then to make a native module that listens for all
> hooks and delegates processing to design documents that indicate they
> handle certain hooks. In other words, build the un-privileged module
> system on top of the privileged one. Let administrators worry about
> file systems when installing the privileged ones. It's really not so
> bad. Sysadmins have been installing things for years, and actually
> probably prefer to install stuff on the command line than with a POST
> to /_replicate. The couch-config utility is there to facilitate this.
> What we probably need is an extra location for user-installed add-on
> applications that need to be added to the Erlang library path and used
> for loading configured applications.
>>        - Futon User Interface to the installed modules, with options to enable,
>> disable, uninstall, rate and configure
> Yes again. I like the gist, but I don't know if it should be bound up
> in Futon or not.
>>        - The tools for installation/uninstallation and enabling/disabling as
>> well as packaging and dependency resolution
> That sounds like figuring out a good way to structure the
> configuration sections for modules. See the JIRA tickets I referenced
> for some discussion of that.
>>        Open questions: What existing projects could be made use of, maybe there
>> is something great for the general erlang world, maybe there is a project
>> like zero install, that         could be made use of, maybe that is not
>> necessary, what are your thoughts?
> I would also check out Agner:
> Basically, if we could get to the point where CouchDB administrators
> could use Agner to install new pieces I'd consider that a huge win.
> If, in the process, we get a modularized core and support for a
> design-document-level hooks system I'd consider that an epic win. I
> never say epic win. I would say it three times out loud.
>> I wish you all a happy new year!
> Thanks :)
> -Randall

View raw message