incubator-bloodhound-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Martin <gary.mar...@wandisco.com>
Subject Re: [BEP-0003] Misc questions
Date Mon, 17 Dec 2012 10:58:17 GMT
Sorry it took me so long to reply.

On 14/12/12 11:25, Jure Zitnik wrote:
> On 12/13/12 4:17 PM, Gary Martin wrote:
>> On 13/12/12 13:41, Jure Zitnik wrote:
>>> The idea we are playing with is that in addition to translating 3rd 
>>> party plugin DMLs (SELECT/INSERT/UPDATE/DELETE) (to present plugins 
>>> with a view of trac/bh resources as seen from the current product 
>>> scope), the DDLs (CREATE/ALTER/DROP) for the custom tables should 
>>> also be translated to support product scope(s) - this would be 
>>> accomplished by creating per product custom table(s), prefixing the 
>>> table name with product prefix. This (in combination with the 
>>> currently implemented SQL translation) would present product unaware 
>>> 3rd party plugins with product scoped tables for both, trac/bh 
>>> tables and any custom table the plugin would create. References 
>>> between the tables would also work and the content of the tables 
>>> would always represent data based on the resources in the product 
>>> scope.
>>>
>>> Naive approach would be to solve 3rd party custom table the same way 
>>> as trac/bh tables (by adding product column). This does not work for 
>>> two reasons:
>>> - schema upgrades - if the plugin chooses to upgrade it's custom 
>>> table schema the usual way of doing this is to copy data to temp 
>>> table, drop original table (this would effectively drop data for all 
>>> products), recreate new table with changed schema and fill it from 
>>> temp table
>>> - table schema changes - not really sure how to implement ALTER 
>>> TABLE if we modify the original schema (connected with the first 
>>> reason)
>>>
>>> To summarize the idea:
>>>
>>> 1. for 3rd party plugins that are product unaware, any custom table 
>>> being created is namespaced to the product by prefixing the table 
>>> name with the product name (in a similar way as discussed resource 
>>> name columns). SQL translator functionality will need to be extended 
>>> to support DDL.
>>>
>>> 2. the SQL translator will be changed in such a way that it will 
>>> support the following table 'types':
>>>    - non-translated tables - tables that need no translation 
>>> (session, cache, attachments,...)
>>>    - trac/bh tables with product scope - these are tables with 
>>> product specific resource (enum, component, ticket, milestone, 
>>> version,...) - the product scope for these tables is implemented 
>>> using 'product' column
>>>    - 3rd party, product-unaware plugin custom tables - product scope 
>>> for these tables is implemented by prefixing the table name with 
>>> product prefix
>>>
>>> 3. changes to product unaware plugin install/upgrade process - 
>>> install/upgrade (IEnvironmentSetupParticipant) would need to be 
>>> invoked for all currently defined products (within that product 
>>> scope of course)
>>>
>>> 4. adding a new product would need to invoke 3rd party (product 
>>> unaware) plugin installation
>>>
>>> How does that sound to everyone?
>>
>> Thanks.. we definitely needed this to evaluate the solution properly!
>>
>> I don't strictly mind what solution is implemented as long as we are 
>> not going too far down a dead end. I would also generally prefer 
>> whichever solution turned out to be simplest but that will be far 
>> harder to judge.
>>
>> Interestingly, if you are also looking at managing table names, it is 
>> not inconceivable that you could add the product fields to the 3rd 
>> party resource tables after all, as long as you were also prepared to 
>> discover which fields should be unique together with product. One of 
>> the possible dangers with this might be that a plugin is at some 
>> point allowed to adjust the schema (obviously we wouldn't mean to 
>> allow it - it would still be possible though) so it still might be 
>> wise to change the table name anyway. There is the possible advantage 
>> that it limits the differences in processing a little.
>>
>
> That (adding product column to 3rd party resources) was one of the 
> ideas that were evaluated. The problem with that approach is the 
> schema upgrade process that usually goes like this (even in trac itself):
> 1. copy data to temp table
> 2. drop original
> 3. create new table with changed schema
> 4. fill new table from temp table & drop temp table
>
> The problem is step 2 (table drop) as there is no way of preventing 
> plugin from dropping the table regardless of the fact that the table 
> holds data for all product scopes and not just the current one.

I still don't see that as hugely different from translating the initial 
creation of the table but there may well be more to keep track of than I 
appreciate. The plugin upgrading all product scopes in a single step 
does not seem to be the biggest issue as long as the product column is 
maintained in each operation. Requirements that a plugin has on 
sequences may also cause issues of course.

>
>> There is actually one final thing that seems to have been left 
>> without any consideration, and that is whether we can get away with 
>> allowing third party resources to be available across all scopes. I 
>> assume that the initial implementations will start off this way anyway!
>>
>
> If we take the 'table name prefix' approach, 3rd party resource will 
> only be available in current product scope. Only product aware plugins 
> and bloodhound code will be able to access resources through all 
> scopes using the global database connection that will have the SQL 
> translator turned off. The interesting issue here is how to present 
> that 3rd party resource (scattered across multiple tables) through 
> global scope without requiring them to do UNION SELECTs on all tables. 
> We might provide helper functions for that though ... something to 
> think about but definitely not critical at the moment.

Well, what I was wondering is whether we would have something useful at 
an intermediate stage if we only have product scoping on non-3rd party 
resources. That would allow the initial implementation to ignore the 
issues of the table name prefix which could slow our adoption of the 
solution while all the work is in progress.

However, the global scope is certainly an interesting problem in the 
multi-table solution. It may help if we assume that plugins would have 
to be product-aware to have certain abilities in global scope but I have 
no doubt that there will be plenty of complicating factors.

Cheers,
     Gary

Mime
View raw message