Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of kxepal@gmail.com designates
 209.85.215.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJNb-9pQmqPYCT3v_QXH-W_ZtGQwXmMrcXPCwHTimvc7x12_QA@mail.gmail.com>
References: <40BCC84F-C695-492E-B56E-08C60955A3D8@apache.org>
	<CAHdjipK3UOg3FbnA_sGUpcP8YXOa0PYfnn3Zx0JmTodPjJdu+A@mail.gmail.com>
	<CAJNb-9pQmqPYCT3v_QXH-W_ZtGQwXmMrcXPCwHTimvc7x12_QA@mail.gmail.com>
Date: Thu, 1 Nov 2012 09:58:37 +0300
Message-ID: 
 <CAHdjipLuuKa5R3_B4zd_CsA-f2PmR9uXS2DiDM+gbW3GFJPZVA@mail.gmail.com>
Subject: Re: CouchDB Plugins First Draft
From: Alexander Shorin <kxepal@gmail.com>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi, Benoit!

> - installation and upgrade via HTTP

You'd remind me one thing:

http://davispj.com/2010/09/26/new-couchdb-externals-api.html

Could this plugins be just one shoot wrapper for proxy with external
process / os_daemon setup? I see there is only need to let them
announce to CouchDB what they can do or handle some part of CouchDB
functionality (e.g. slots concept).
--
,,,^..^,,,


On Thu, Nov 1, 2012 at 10:49 AM, Benoit Chesneau <bchesneau@gmail.com> wrot=
e:
> On Wed, Oct 31, 2012 at 11:28 PM, Alexander Shorin <kxepal@gmail.com> wro=
te:
>
>> Hi Jan.
>>
>> More detailed and explained question from IRC meeting.
>>
>> Why there is need to reinvent own package manager when most modern
>> systems (even Windows) already has package manager?
>>
>>
>> What problem tried to be solved?
>> ------------------------------------------------------------
>>
>> Easy find, install and activate any CouchDB plugin in one-shot action,
>> right? Following example from gist:
>>
>> $ cpm # couch plugin manager
>> $ cpm search geocouch
>> $ cpm install geocouch
>>
>> While it tries to simplify extending of CouchDB, it brings a lot of
>> problems into system
>>
>>
>> What problems cpm brings?
>> -----------------------------------------------------
>>
>> 1. Dependencies.
>>
>> While any system package already able to resolve dependencies,
>> conflicts, provide slots for different versions, cpm will need to
>> reimplement this features. Geocouch isn't good example, next one is
>> much better:
>>
>> $ cpm install couchdb-lucene # what about dependencies?
>>
>> I don't think (and hope it will not) that this command will install
>> JRE and other java things into my system while I'm only expects this
>> plugin. System package managers already know what need and which
>> versions of to properly install lucene. Should this `cpm install`
>> command only check for suitable environment before landing? Or may be
>> cpm will know how to cooperate with system package managers to propose
>> install required dependencies?
>>
>> 2. Namespace.
>>
>> Since CouchDB installs system-wide (most cases) should plugins be
>> installed in same way? Or each user may have his own set of active
>> plugins? But it looks like that only CouchDB admin may operate with
>> plugins from Futon, so plugins have to live at least by path
>> accessiable for couchdb user. So there couldn't be personalised
>> plugins and npm system wouldn't work. So, cpm utility is only useful
>> for root user which is already has access to system package manager.
>>
>> 3. BigCouch
>>
>> How plugins system proposed to be work after BigCouch merge? Would
>> plugins be installed for all shards or only just for current one? Will
>> cluster works right if some shards would have geocouch-1.0, some of
>> them has geocouch-1.3 with breaking changes and for others it just
>> missed?
>>
>> Why system package manager?
>> ---------------------------------------------------------
>>
>> $ flaggie dev-db/couchdb +lucene # or echo "dev-db/couchdb lucene" >>
>> /etc/portage/package.use
>> $ emerge couchdb
>>
>> And we'd got CouchDB with couchdb-lucene plugin and handled all
>> required dependencies. Now system takes care about packages
>> consistency and we're always sure and accidental iceadtea uninstall
>> would be passed silently since it have to break our couchdb-lucene
>> extension.
>>
>> In same way other projects are already have implemented their
>> "plugins" system: nginx, php. Yes, they have not plugins, but modules,
>> but since they acts on system level there is no easy and secure way to
>> let everyone enable/disable/install any thirdparty component and this
>> way is the right one.
>>
>> Following RabbitMQ project, they ships with all main plugins on board
>> - you need just to enable with through cli utility. If you need to
>> install some third party plugin, just copy his directory to rabbits
>> plugins one. But to be honest, RabbitMQ plugin manager tool doesn't
>> tries to act as package manager: it only shows, enables and disables
>> plugins, nothing more - that's why their approach works fine.
>>
>>
>> So I've smoothly went from original question to set of other ones, but
>> I suppose they are also interested to be answered since they are
>> defined show cpm will work and handle plugins.
>>
>> P.S. Sorry for poorly mind dump style, 2:40 am there. I could try to
>> explain something in other way if it wasn't clear enough...
>>
>> --
>> ,,,^..^,,,
>>
>>
>> On Mon, Oct 29, 2012 at 8:41 PM, Jan Lehnardt <jan@apache.org> wrote:
>> > Hey all,
>> >
>> > during one of the past IRC Meetings I promised to write up
>> > what I have in mind for a CouchDB Plugin System.
>> >
>> > Here is my first draft:
>> >
>> >   https://gist.github.com/3974676
>> >
>> > This is very early stage, but I wanted to get the idea out
>> > of the door. If anyone feels compelled to go into further
>> > detail, or even implement some of this, more power to you!
>> >
>> > Note: This is especially interesting for the non-Erlang
>> > folks here that need an excuse to contribute :)
>> >
>> > For the frontend, we=E2=80=99d need some mockups on how the Futon
>> > interaction would work (that is regardless of the state
>> > of Futon.Current and Futon.Next).
>> >
>> > On the backend, you could start figuring out what the
>> > registry could look like.
>> >
>> > I=E2=80=99m looking forward to your feedback!
>> >
>> > Cheers!
>> > Jan
>> > --
>> >
>>
> I second that , we need to find a good way to handle plugin.  I think we
> have 4 cases to solve reading the gist:
>
> 1. upload from the HTTP api an Erlang plugin.
> 2. install a couchapp via HTPP
> 3. install an erlang plugin  from the fs
> 4. install a couchapp from the fs
>
>
> Both 2 & 4 can be easily handled today. Though it raises
> some interesting problems:
>
> - how to allow a user to push a couchapp
> - how to make sure that the couchapp can have access to a specific db on
> the node and the cluster.
>
> 1 & 3 have some other questions:
>
> - how is done the build? Is the plugin sent as a binary or is build on th=
e
> target machine?
> - how to fix dependencies ? In that case , the requirements are generally
> started when the plugin start and simply fail if the requirement isn't
> found. This is what do elasticsearch or rabbitmq for ex.
> - What if a plugin is installed while the node is running ? Should we
> handle hot upgrade?
> - What happen when the plugin is unloaded or unregistered. Especially if =
it
> depends on some dependencies.
> - sandboxing. How to make sure a "native" plugin won't do bad things on m=
y
> system
> - how to deploy the plugin on a cluster?
>
> Imo we should have a look on how it's handled in the mobiles right now:
>
> - get an app from a location
> - check it's security properties
> - check requirements
> - ask to the authorized users if the app can have access to a db/resource
> ...
>
> But, hat if we distinct backend plusgins in erlang from the other. In my
> understanding we need both to extend internals of couchdb. But maybe we c=
an
> find to another system allowing us to extend couchdb. Such plugin could b=
e
> then written in javascript, lua, elixir ... And could be sandboxed.
>
> This is actually something I am working on and I will release preliminary
> work at the end of the week or smth like it but in short the idea I have =
is
> to extend the couchapp system for internal couchapps which are executed i=
n
> couchdb instead of the browser and have access to the full api. Networks
> access & disk are limited  except if specifically allowed. Using such
> features allows :
>
> - installation and upgrade via HTTP
> - plugin replication: you don't have to install cheff, puppet... plugins
> are contained in a doc and can be replicated like others.
> - the security system of the frontend couchapps could be reused
>
> Anyway I quite like the description of plugins described by jan  and I am
> happy to help on that . Btw why not using the wiki instead of github to
> handle the spec writing?
>
> - benoit