Mailing-List: contact bloodhound-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: bloodhound-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of gary.martin@wandisco.com
 designates 209.85.214.47 as permitted sender)
Message-ID: <5076ECD2.6020103@wandisco.com>
Date: Thu, 11 Oct 2012 16:59:14 +0100
From: Gary Martin <gary.martin@wandisco.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:15.0) Gecko/20120912 Thunderbird/15.0.1
MIME-Version: 1.0
To: bloodhound-dev@incubator.apache.org
Subject: Re: Tests and CI (was: A word on Release 2)
References: 
 <CAPt9QwQdaC+cEwVRe__CxhX7vvqk40Uckc3bDBVn5y+K6e5VZQ@mail.gmail.com>
 <CAGMZAuOpvdRPBTwkv7LgWUixdg5qttTxGq71UBbrsVi=V7mY2Q@mail.gmail.com>
 <CAPt9QwTXEcCDLoF67AANki_LCYtKFqSW583KTE3S_X1Uq-3uig@mail.gmail.com>
 <CAKE9Bs8seSW4hM=gV+RnyFhBtJJMGLE3JvFFLtUPH75MbY4pmQ@mail.gmail.com>
 <CAPt9QwTuq+uKQM979XcbxYixbsR1NN1JGEgsCRAB_vWg3+Kb+Q@mail.gmail.com>
 <CAGMZAuNO-K1imLE7F-pcRu-yDw5FGOJb-KM+6MENAqiXvbpwWQ@mail.gmail.com>
 <CAKE9Bs-_OSY4+Tu9jhR1XzdEVwGmP4Czy3-0n3QBXWJ7K2O8WA@mail.gmail.com>
 <50760EFB.1030504@wandisco.com> <507631E3.5080808@wandisco.com>
In-Reply-To: <507631E3.5080808@wandisco.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 11/10/12 03:41, Branko Čibej wrote:
> On 11.10.2012 02:12, Gary Martin wrote:
>> On 10/10/12 18:26, Peter Koželj wrote:
>>> I admit it, some of my fears could just be the cause of the fact that
>>> I am
>>> not familiar with Trac plugin architecture and the Trac plugin
>>> ecosystem as
>>> a whole yet.
>>>
>>> Just a few use cases for quick "does it float" test:
>>> 1. Let's say we adopt some plugins for multiproducts, ticket relations,
>>> data import/export and so on...
>>> 2. Then we want to add user (admin) defined ticket relation per product
>>> 3. And then we want to support custom ticket workflows per product
>>> and per
>>> ticket type
>>> 4. And now we want the ticket import/export from/to Jira to work across
>>> Trac and all other plugins (including ticket relations, multiproducts
>>> and
>>> custom per product workflows plugins)
>>> 5. Add white labeling across Trac and all other plugins
>>> 6. Implement new search query language that will understand tickets,
>>> products and relations
>>> 7. Add REST API's for all of the above
>>> 8. ...
>>>
>>> Now, all this plugins (products, relations, import, export, search,
>>> white
>>> labeling, new search...) will either have to know filthy little details
>>> about each other (complex inter-plugin dependency) or some super heavy
>>> shifting in the plugin interface will be necessary. And that super heavy
>>> shifting will have to be pushed back to Trac (if it wants it) and
>>> only then
>>> to these plugins or create new Bloodhound plugin API on top of Trac
>>> plugin
>>> API and then request plugin authors to support it.
>>>
>>> Am I missing something?
>> I am not sure that I see the same difficulties. It seems a pretty
>> normal thing for a program to be divided into components in such a way
>> that keeps filthy details hidden. All we are doing here is allowing
>> the set of components to be distributed between core trac, bloodhound
>> plugins and external plugins. This is achieved through Trac and
>> plugins defining interfaces, with components then being able to
>> implement the interfaces and/or iterate through components that
>> provide the interface.
>>
>> So far I have found little reason to put any "super heavy shifting"
>> into trac itself as they appear to have provided enough interfaces to
>> allow for some fairly complex subverting of standard trac processing.
>> The namespacing for multiproducts is a case in point where, so far, it
>> appears to be achievable outside of trac. If it is not possible to do
>> all that we want to with, say, per-product configuration or workflow,
>> it might be that we will need to introduce a new interface to Trac to
>> allow us to take over processing at the appropriate time but I am
>> hoping that this will not be necessary.
> The point Peter is trying to make is this: if we take as an example the
> multiproduct plugin:
>
> This plugin has to change the trac database schema (or rather, add a
> more-or-less independent namespace mapping to the existing schema).
> Consequentially, plugins must be allowed to either update the core
> schema, or create their own additional schema. Whichever it is affects
>
>    * database schema upgrades (from any older version to any newer version)
>        o either each plugin has to have its own schema upgrade mechanism
>          -- a nightmare, or
>        o the plugin API must define unified schema management protocols,
>          wich /all/ plugins that fiddle with the schema have to conform to

Whether it is a nightmare or not, the responsibility for schema upgrade 
is on the shoulders of the plugin. If we want to make this a more 
uniform experience, and if there is no solution within trac, we would 
probably have to create a plugin that would provide these services and 
hope that the plugins we want to use will adopt this as a dependency. Of 
course, it would be essential that this plugin stays compatible with 
trac and it would effectively need to remain distributable as a 
separable entity from the rest of bloodhound unless we can convince 
upstream trac of the need for this early on.

>    * data export/import
>        o as above, either each plugin handles its own export/import (and
>          incidentally defines its own export format), which implies that
>          anyone who wants to write an exporter from another issue
>          tracking system to Bloodhound's export format has to know
>          exactly which plugins are in fact part of core Bloodhound; or,
>        o the plugin API must define a common event streaming protocol
>          that can be converted to a generic export format; which does not
>          seem trivial to me.
>
> As another example, a "new search" plugin would conceivably allow
> defining event notification triggers based on the results of a custom
> query having changed. Any export format has to have a way to represent
> this (if only because JIRA has this feature and you'd eventually want to
> be able to import these definitions if BH achieves feature parity in
> that respect).
>
> Almost every new feature that Bloodhound adopts and implements via a
> plugin affects with the goal of creating a common export format, a
> concept that trac does not have and therefore isn't likely to be
> supported in the plugin API.

For me your argument continues to suggest that this is a problem for all 
plugins which it is not.

The only problem is with external plugins as they may not want to 
implement the interfaces that we want them to, or indeed implement them 
correctly. We have to evaluate external plugins to see what problems 
there are and decide what matters. I don't want us to cultivate a not 
invented here attitude whilst we are able to benefit from the work of 
others and can attempt to provide reciprocal benefits.

Cheers,
     Gary