incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Minimally Viable Plugins
Date Fri, 02 Aug 2013 23:56:32 GMT

On Aug 3, 2013, at 00:02 , Russell Branca <chewbranca@apache.org> wrote:

> This is fantastic, Jan! Glad to see this coming along.
> 
> One of the goals with Fauxton has always been to make it easy for plugins
> to extend the interface and provide new functionality. I've been toying
> with the idea of having a _fauxton db that plugins install to as docs with
> attachments, but that's for a different thread. The general idea here is
> that a plugin will be able to extend Fauxton by adding a new page with it's
> own functionality, or hook into existing pages to extend other areas.
> 
> For instance, you could have a couchdb-lucene plugin that hooks into the
> databases list and allows you to add interfaces for building full text
> indexes and searching on existing indexes. Or you could have a dedicated
> page for Geocouch, or whatever.
> 
> The functionality is there, but it's still a bit of a manual process, so
> we'll need to make it more dynamic and smooth out the rough edges.
> 
> I'm very excited to see progress being made on plugins, great work!

Thanks, I’m glad you like this! :)

Another way to get the Fauxton plugin loaded would be to extend the
/_plugins API endpoint, so Fauxton could request GET /_plugins/<pluginname>/
and it would serve <couchdblibdir>/plugins/<pluginname/priv/www which is
just a place for Fauxton-enabled plugins.

Fauxton would walk /_config/plugins/ to get to a list of plugins.

In fact that should be pretty simple to set up.

For now I am trying to avoid having a custom database for this, mostly
because I don’t think there are many advantages (e.g. replication of
plugins?) and code complexity. These priorities might change in the
future, but for now I am happy to get this working at all :)

If you are okay with the above plan of serving plugin HTML/JS/CSS from
/_plugins/<pluginname>, I’m happy to add this to the branch.

Best
Jan
-- 



> 
> 
> -Russell
> 
> 
> On Fri, Aug 2, 2013 at 2:17 PM, Jan Lehnardt <jan@apache.org> wrote:
> 
>> And a few more (from COUCHDB-1867):
>> 
>> - Add uninstall, incl. Futon UI.
>> - Only install a plugin if the source and target CouchDB version matches.
>> - Rebase against master.
>> 
>> * * *
>> 
>> This concludes my list for a Minimally Viable Plugin feature. (See the
>> original email or README.md* for the roadmap)
>> 
>> I’d appreciate some more reviews & feedback**, but other than that, I’d be
>> happy to ship this as an experimental feature in any next release.
>> 
>> *
>> https://github.com/janl/couchdb/blob/1867-feature-plugins/src/couch_plugins/README.md#roadmap
>> **
>> https://github.com/janl/couchdb/compare/apache:master...1867-feature-plugins
>> 
>> 
>> Best
>> Jan
>> --
>> 
>> On Aug 1, 2013, at 19:34 , Jan Lehnardt <jan@apache.org> wrote:
>> 
>>> A few updates:
>>> 
>>> By Bob Ippolito / @etrepum:
>>> - Plugins are now installed in libdir (instead of /tmp).
>>> - Config loading is now done with proper .ini files.
>>> - Various cleanups and code review (Thanks!).
>>> 
>>> Mine (most suggested by Bob):
>>> - `plugins.html` now shows you if a plugin is already installed.
>>>  and which version, if it doesn’t match the installable one.
>>> - The Install button now disables after an installation.
>>> - Plugins are now registered with couch_config as
>>>  /_config/plugins/name = version
>>> - Updated `couch-config` to print --erlang-version and --erl-bin
>>> - Updated the geocouch plugin to use the new options in
>>>  `couch-config`.
>>> - Added Bob Ippolito’s couchperuser plugin to Futon.
>>> 
>>> 
>>> Best
>>> Jan
>>> --
>>> 
>>> 
>>> 
>>> On Jul 31, 2013, at 19:07 , Jan Lehnardt <jan@apache.org> wrote:
>>> 
>>>> Heya,
>>>> 
>>>> I couldn’t help myself thinking about plugin stuff and ended up
>>>> whipping up a proof of concept.
>>>> 
>>>> Here’s a <1 minute demo video:
>>>> 
>>>> https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.mov
>>>> 
>>>> Alternative encoding:
>>>> 
>>>> https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.m4v)
>>>> 
>>>> 
>>>> In my head the whole plugin idea is a very wide area, but I was so
>>>> intrigued by the idea of getting something running with a click on a
>>>> button in Futon. So I looked for a minimally viable plugin system.
>>>> 
>>>> 
>>>> ## Design principles
>>>> 
>>>> It took me a day to put this all together and this was only possible
>>>> because I took a lot of shortcuts. I believe they are all viable for a
>>>> first iteration of a plugins system:
>>>> 
>>>> 1. Install with one click on a button in Futon (or HTTP call)
>>>> 2. Only pure Erlang plugins are allowed.
>>>> 3. The plugin author must provide a binary package for each Erlang (and,
>>>> later, each CouchDB version).
>>>> 4. Complete trust-based system. You trust me to not do any nasty things
>>>> when you click on the install button. No crypto, no nothing. Only
>>>> people who can commit to Futon can release new versions of plugins.
>>>> 5. Minimal user-friendlyness: won’t install plugins that don’t match
>>>> the current Erlang version, gives semi-sensible error messages
>>>> (wrapped in a HTTP 500 response :)
>>>> 6. Require a pretty strict format for binary releases.
>>>> 
>>>> 
>>>> ## Roadmap
>>>> 
>>>> Here’s a list of things this first iterations does and doesn’t do:
>>>> 
>>>> - Pure Erlang plugins only. No C-dependencies, no JavaScript, no
>> nothing.
>>>> - No C-dependencies.
>>>> - Install a plugin via Futon (or HTTP call). Admin only.
>>>> - A hardcoded list of plugins in Futon.
>>>> - Loads a pre-packaged, pre-compiled .tar.gz file from a URL.
>>>> - Only installs if Erlang version matches.
>>>> - No security checking of binaries.
>>>> - No identity checking of binaries.
>>>> 
>>>> Here are a few things I want to add before I call it MVP*:
>>>> 
>>>> - Uninstall a plugin via Futon (or HTTP call). Admin only.
>>>> - Only installs if CouchDB version matches.
>>>> - Binaries must be published on *.apache.org.
>>>> - Register installed plugins in the config system.
>>>> - Make sure plugins start with the next restart of CouchDB.
>>>> - Show when a particular plugin is installed.
>>>> 
>>>> *MVP hopefully means you agree we can ship this with a few warnings
>>>> so people can get a hang of it.
>>>> 
>>>> Here is a rough list of features squared against future milestones:
>>>> 
>>>> Milestone 2: Be creator friendly
>>>> - Make it easy to build a CouchDB plugin by providing one or more easy
>>>> to start templates.
>>>> - Make it easy to publish new plugins and new versions of existing
>> plugins.
>>>> - Make it easy to supply packages for multiple Erlang & CouchDB
>> versions.
>>>> 
>>>> Milestone 3: Public registry
>>>> - Instead of hardcoding a list of plugins into Futon/Fauxton, we load
>>>> a list of applicable plugins from a central (and configurable)
>>>> plugins repository.
>>>> - This allows plugin authors to publish new plugins and new versions
>>>> of existing plugins independently.
>>>> 
>>>> Milestone 4: Other Languages
>>>> - Figure out how to handle C-dependencies for Erlang plugins.
>>>> - Figure out how to allow other language plugins
>>>> (c.f. non-JS query servers)
>>>> 
>>>> Milestone X: Later
>>>> - Add some account/identity/maybe crypto-web-of-trust system for
>>>> authors to publish “legit” plugins.
>>>> - Sign & verify individual releases.
>>>> 
>>>> A few more things that can happen concurrently depending on what
>>>> plugins require:
>>>> - Integrate Erlang/JS tests in the installation
>>>> - Integrate docs
>>>> 
>>>> 
>>>> ## How it works
>>>> 
>>>> This plugin system lives in `src/couch_plugins` and is a tiny CouchDB
>>>> module.
>>>> 
>>>> It exposes one new API endpoint `/_plugins` that an admin user can
>>>> POST to.
>>>> 
>>>> The additional Futon page lives at /_utils/plugins.html it is
>>>> hardcoded.
>>>> 
>>>> Futon (or you) post an object to `/_plugins` with four properties:
>>>> 
>>>>  {
>>>>    "name": "geocouch", // name of the plugin, must be unique
>>>>    "url": "http://people.apache.org/~jan", // “base URL” for plugin
>> releases (see below)
>>>>    "version": "couchdb1.2.x_v0.3.0-11-gd83ba22", // whatever version
>> internal to the plugin
>>>>    "checksums": {
>>>>      "R15B03": "ZetgdHj2bY2w37buulWVf3USOZs=" // base64’d sha hash
>> over the binary
>>>>    }
>>>>  }
>>>> 
>>>> `couch_plugins` then attempts to download a .tar.gz from this
>>>> location:
>>>> 
>>>> 
>> http://people.apache.org/~jan/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03.tar.gz
>>>> 
>>>> It should be obvious how the URL is constructed from the POST data.
>>>> (This url is live, feel free to play around with this tarball).
>>>> 
>>>> Next it calculates the sha hash for the downloaded .tar.gz file and
>>>> matches it against the correct version in the `checksums` parameter.
>>>> 
>>>> If that succeeds, we unpack the .tar.gz file (currently in `/tmp`,
>>>> need to find a better place for this) and adds the extracted directory
>>>> to the Erlang code path
>>>> 
>> (`code:add_path("/tmp/couchdb_plugins/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03/ebin")`)
>>>> and loads the included application (`application:load(geocouch)`).
>>>> 
>>>> Then it looks into the `./config` directory that lives next to `ebin/`
>>>> in the plugin directory for a file `config.erlt` (“erl-terms”). with
a
>>>> list of configuration parameters to load. We parse the file and set
>>>> the config directives one by one.
>>>> 
>>>> If that all goes to plan, we report success back to the HTTP caller.
>>>> 
>>>> That’s it! :)
>>>> 
>>>> It’s deceptively simple, probably does a few things very wrong and
>>>> leaves a few things open (see above).
>>>> 
>>>> One open question I’d like an answer for is finding a good location to
>>>> unpack & install the plugin files that isn’t `tmp`. If the answer is
>>>> different for a pre-BigCouch/rcouch-merge and post-BigCouch/rcouch-
>>>> merge world, I’d love to know :)
>>>> 
>>>> 
>>>> ## Code
>>>> 
>>>> The main branch for this is 1867-feature-plugins:
>>>> 
>>>>   ASF:
>> https://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=log;h=refs/heads/1867-feature-plugins
>>>> GitHub: https://github.com/janl/couchdb/compare/1867-feature-plugins
>>>> 
>>>> I created a branch on GeoCouch that adds a few lines to its `Makefile`
>>>> that shows how a binary package is built:
>>>> 
>>>> 
>> https://github.com/janl/geocouch/compare/couchbase:couchdb1.3.x...couchdb1.3.x-plugins
>>>> 
>>>> * * *
>>>> 
>>>> I hope you like this :) Please comment and improve heavily!
>>>> 
>>>> Let me know if you have any questions :)
>>>> 
>>>> If you have any criticism, please phrase it in a way that we can use
>>>> to improve this, thanks!
>>>> 
>>>> Best,
>>>> Jan
>>>> --
>>>> 
>>> 
>> 
>> 


Mime
View raw message