Mailing-List: contact bloodhound-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: bloodhound-dev@incubator.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Message-ID: <50C9DB0F.8020705@digiverse.si>
Date: Thu, 13 Dec 2012 14:41:35 +0100
From: Jure Zitnik <jure@digiverse.si>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:17.0) Gecko/17.0 Thunderbird/17.0
MIME-Version: 1.0
To: bloodhound-dev@incubator.apache.org
Subject: Re: [BEP-0003] Misc questions
References: 
 <CAJSKAsghYs7e6rmv7-04y-KmBMCto0RjeFBSaOfM3yXyjRd=mQ@mail.gmail.com>
 <50C886DE.2090007@wandisco.com> <50C8A043.3050804@digiverse.si>
 <50C8B156.3050704@wandisco.com>
In-Reply-To: <50C8B156.3050704@wandisco.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

On 12/12/12 5:31 PM, Gary Martin wrote:
> On 12/12/12 15:18, Jure Zitnik wrote:
>>
>>>> 2. Something that we might forgot: What about 3rd party plugin 
>>>> tables that
>>>> reference multiproductized Trac tables?
>>>>      Will probably need to proclaim these incompatible when more 
>>>> than one
>>>> product is in effect?
>>>
>>> Good point. To keep track of records from tables from third party 
>>> plugins, this approach doesn't quite work. I would have thought that 
>>> we would be better off using a separate table to keep track of the 
>>> resources that belong to a product. Is this another area that has 
>>> not been updated based on discussions?
>>
>> The current SQL translator implementation would show 3rd party 
>> plugins a view of translated tables that would only include resources 
>> from the currently selected product scope. If the plugin makes a 
>> reference to a resource by it's name everything should work fine as 
>> the reference would be consistent each time when in that specific 
>> product environment (as the plugin would always get the same view of 
>> the database).
>>
>> Things start breaking if there's a resource with the same name in 
>> multiple products, unless the translator is changed to return names 
>> with product namespace being prefixed to the actual resource name for 
>> example. The plugins would get version name 'BH:1.0' instead of '1.0' 
>> for example. Still, this doesn't solve the problem entirely as the 
>> plugin (that's not aware of products) would end up (in it's own 
>> tables) with references to different resources from different 
>> products and maybe that's not exactly what's expected to happen...
>>
>> Keeping track of resource belonging to a product using a separate 
>> resource mapping table also unfortunately doesn't solve the issue. 
>> We'd need to change the schema anyway as in the current database 
>> model, all tables have 'name' column as their key. We could of course 
>> reference the same resource from different products using the 
>> separate mapping table but we'd be referencing the same record and 
>> changing the name of that record would change the resource in all 
>> products which is, at least imo, not what we want.
> Ah yes, I forgot about my ideas for that. For the purposes of unique 
> keys I was thinking of including some kind of prefix as part of the 
> name - not necessarily the product namespace as we could consider it 
> better to leave this as a constant with a means to link the prefix to 
> the namespace.
>

As I wrote above, only prefixing names doesn't solve the issue. If we 
have a product unaware plugin, that plugin should only see resources 
from the current product scope and this (for trac/bh resources) is 
currently accomplished by translating SQLs in such a way that the plugin 
only sees a product specific view of resources. The problem arises when 
the plugin stores references to that resources in their own tables as it 
might end up (unless we do something about it) with resource mixed from 
different products.

Let's say we have a custom table named 'sorted_milestones' with columns 
'milestone_name' and 'sequence' and database constraint that 'sequence' 
is unique. If plugins sees only resources (in this case milestones) from 
one product at a time, constraint will fail as logic sees only product 
scope specific part of milestones. When product scope would change, the 
plugin sees completely different set of milestones and has no way of 
knowing that certain sequences are already 'taken'.

On the other hand, assuming we only prefix resources without filtering 
them for multi-product unaware plugins would cause the user interface 
for those plugins to show everything (every resource defined regardless 
of product scope). In addition to that, it'd be hard (if not impossible) 
to remove the prefix before showing that to the user. Also, it's not 
exactly obvious that we can remove the prefix before passing that to 
plugins/UI as we never know how those will be used further down the line...


> The problem with just adding fields to each model is not so much a 
> problem from the point of view of 3rd party plugins accessing those 
> models that are modified in such a way, but rather with those resource 
> tables that are added by the third party plugins. 

Completely agree.

> These would have to be modified to add the product to their tables 
> too. Is the suggestion that we do that modification for externally 
> defined resources or only provide the ability for specific plugins?
>

The idea we are playing with is that in addition to translating 3rd 
party plugin DMLs (SELECT/INSERT/UPDATE/DELETE) (to present plugins with 
a view of trac/bh resources as seen from the current product scope), the 
DDLs (CREATE/ALTER/DROP) for the custom tables should also be translated 
to support product scope(s) - this would be accomplished by creating per 
product custom table(s), prefixing the table name with product prefix. 
This (in combination with the currently implemented SQL translation) 
would present product unaware 3rd party plugins with product scoped 
tables for both, trac/bh tables and any custom table the plugin would 
create. References between the tables would also work and the content of 
the tables would always represent data based on the resources in the 
product scope.

Naive approach would be to solve 3rd party custom table the same way as 
trac/bh tables (by adding product column). This does not work for two 
reasons:
- schema upgrades - if the plugin chooses to upgrade it's custom table 
schema the usual way of doing this is to copy data to temp table, drop 
original table (this would effectively drop data for all products), 
recreate new table with changed schema and fill it from temp table
- table schema changes - not really sure how to implement ALTER TABLE if 
we modify the original schema (connected with the first reason)

To summarize the idea:

1. for 3rd party plugins that are product unaware, any custom table 
being created is namespaced to the product by prefixing the table name 
with the product name (in a similar way as discussed resource name 
columns). SQL translator functionality will need to be extended to 
support DDL.

2. the SQL translator will be changed in such a way that it will support 
the following table 'types':
    - non-translated tables - tables that need no translation (session, 
cache, attachments,...)
    - trac/bh tables with product scope - these are tables with product 
specific resource (enum, component, ticket, milestone, version,...) - 
the product scope for these tables is implemented using 'product' column
    - 3rd party, product-unaware plugin custom tables - product scope 
for these tables is implemented by prefixing the table name with product 
prefix

3. changes to product unaware plugin install/upgrade process - 
install/upgrade (IEnvironmentSetupParticipant) would need to be invoked 
for all currently defined products (within that product scope of course)

4. adding a new product would need to invoke 3rd party (product unaware) 
plugin installation

How does that sound to everyone?

Cheers,
Jure