incubator-bloodhound-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jure Zitnik <>
Subject Re: [Apache Bloodhound] #404: Populate default schema on product addition
Date Mon, 25 Feb 2013 17:45:00 GMT
On 2/24/13 7:07 AM, Olemis Lang wrote:
> On 2/22/13, Jure Zitnik <> wrote:
> [....]
>> First, I agree that the environment (global, that is) setup should be
>> apart from product scope.
> I'd agree *IF* upgrades at product scope would be desired .
> Nonetheless I think they will cause a lot of trouble , so I'm strongly
> against them . If we thing in terms of contracts then I strongly
> recommend that the precondition for instantiating product environments
> be «every setup step needed to make components work in product
> contexts has been already performed» ... we'll sleep better at night
> ;)

Upgrades at product scope are not only desired but in my opinion 
actually required (see below). Please note that each custom (3rd party 
plugin) table is created *per product*. Therefor create/upgrade also 
need to be handled per product.

>> It's a bit of a different story on product 'setup'. Now, we agreed that
>> the non multi-product aware components should have a separate view of
>> the database. In addition to system resource tables (components,
>> versions, tickets and such), this view also comprises of custom tables
>> that the component might introduce. These tables are introduced (and
>> later upgraded for that matter) from within IEnvironmentSetupParticipant.
> This is the way I understood your approach [1]_
> {{{
> #!plantuml
> @startuml
> participant "esglobal : EnvironmentSetup" as esglobal
> participant "cglobal : SomeComponent" as cglobal
> participant "esproduct : EnvironmentSetup" as esproduct
> participant "cproduct : SomeComponent" as cproduct
> note over cglobal : cglobal.env = global environment
> note over cproduct : cproduct.env = *some* product environment
> == Global environment upgrade ==
> esglobal -> cglobal : upgrade()
> note left : Global tables as usual
> == Product environment upgrade ==
> esproduct -> cproduct : upgrade()
> note right : Product-specific DB view
> @enduml
> }}}
> If we are talking of such sequences then I'm -1

Yes, if I understood that sequence correctly, that's what I'm proposing 
(assuming cglobal is enabled in esglobal (global) environment only and 
cproduct in 'esproduct' (product) environment).

>> So, if we want to have a separate set of component tables per product
>> (this is what 'separate view of the database' implies),
> Will these tables be extended with product prefix ?
Yes, table names are prefixed with product prefix. As a consequence, 
each non multi-product aware component will end up with as many tables 
as there are products (+1 for null/global one), each table name being 
prefixed with product prefix. There is no other way of doing this as 
there is no way of knowing, what the actual schema/content of the tables is.

>> we should use
>> IEnvironmentSetupParticipant interface and run it within the specific
>> product scope. And this would normally happen when adding product.
>> So
>> from the non multi-product aware component perspective, adding a product
>> would seem much like creating a new environment.
> -1
> that will be the source of many headaches

It is my understanding and opinion that's the only way to move forward, 
excluding headaches ;)

>> Multi-product aware components on the other hand would be handled
>> differently as I described in my mail yesterday...
> I was thinking of anticipating upgrades i.e. something like this [2]_
> {{{
> #!plantuml
> @startuml
> participant "es : EnvironmentSetup" as es
> participant "c1 : SomeComponent" as c1
> participant "c2 : OtherComponent" as c2
> participant ": Multiproductsystem" as mpsys
> note over c1, c2 : Setup participants
> == Global environment upgrade ==
> note over es : In a few words the strategy consists in anticipating
> upgrades needed for MP components.

Not sure what 'anticipating' would mean in this context.

> es -> c1 : upgrade()
> note right : Global tables as usual
> es -> mpsys : upgrade()
> note right
>      1. Upgrades to core tables, if needed
>      2. Upgrades existing plugin tables to multiproduct, if needed
>      3. Replay MP-unaware upgrades performed in this very same upgrade
> loop e.g. c1
> end note
I'm not sure what the 'Replay MP-unaware upgrades' is supposed to mean 
and how to get from 'c1 upgrade' to replay?

Are we supposed to keep log of SQLs that the component executed during 
environment creation/upgrade? I don't think that's doable. Not to 
mention that not only the SQLs would be product specific (references to 
tickets or any other system table for that matter), but the component 
could also be, for example, creating files in env.path.

> es -> c2 : upgrade()
> note right
>      Perform both global as well as multiproduct table upgrades
>      for both MP-aware and MP-unaware components.
> end note
> note over es
>      MP-aware setup participants will only need a single call to upgrade() .
>      'Upgrade actions' being actually product initialization yasks will not
>      be handled at upgrade time, but via product resource listeners instead.
> end note

The latter part (single call to upgrade() for multi-product aware setup 
participants + usage of product resource listeners) is what I proposed 
in one of my previous mails. In my opinion (as I proposed), we will need 
additional interface for multi-product aware components (could be called 

> @enduml
> }}}
> ... and thereby keep all this running in global scope .
I disagree, each product (setup participants) should be upgraded from 
it's own scope. This is required as we want the translator to be running 
in the product scope to translate all tables (SQLs) correctly.

Let's take one step back and take a look at the following use-case.

Let's assume we have a component that is a setup participant. Let's say 
that non multi-product aware components are setup participants when they 
implement 'IEnvironmentSetupParticipant' interface and that 
multi-product aware components are setup participants when they 
implement 'IGlobalSetupParticipant'. Let's also assume (for now) that 
the 'IEnvironmentSetupParticipant' and 'IGlobalSetupParticipant' 
interface's signatures are the same.

Let's say that the 'environment_created' method of the component does 
the following:
1. creates table named 'my_table'
2. inserts into 'my_table' data that aggregates/references some data 
from system tables ('ticket', 'component',...)
3. creates a file in 'env.path' that holds some environment specific data
4. inserts it's database version into 'system' table

Now, let's discuss the relevant scenarios: (global) environment creation 
(install from scratch), product addition and environment upgrade. I'll 
describe scenarios for both cases, first for non multi-product aware 
component and then for multi-product aware component.

1. Global environment creation
-- Non multi-product aware component
In this case, the components 
'IEnvironmetSetupParticipant.environment_created' should be executed in 
global environment only (as there are no products). Result of this would 
be 'my_table' table with aggregated/referenced data from 'null' product 
scope (global scope). Note that the translator does not prefix table 
names when product is 'null'. File that the component creates would be 
stored in the global environment path, the 'system' table would get a 
record with 'null' product prefix.

-- Multi-product aware component
Same as in case of non multi-product aware component, everything happens 
in global environment and through (new) 'IGlobalSetupParticipant' 
interface. Results of 'environment_created' (tables, files, etc) are the 
same as in the case of non multi-product aware component.

2. Product (with 'MYPRODUCT' prefix) addition
-- Non multi-product aware component
Adding a product should invoke component's 
'IEnvironmentSetupParticipant.environment_created' method from within 
'ProductEnvironment('MYPRODUCT')'. This would enable SQL translator in 
that scope with the known rules. Result of this would be 
'MYPRODUCT_my_table' with aggregated/referenced data from 'MYPRODUCT's 
scope. File would be stored in 'MYPRODUCT' product's path. 'system' 
table would get a record with 'MYPRODUCT' product prefix.

-- Multi-product aware component
Multi-product aware component is notified of product addition through 

3. Environment upgrade
-- Non multi-product aware component
Following the normal process (in 'trac.env.Environment'), each non 
multi-product setup participant should be upgraded (if required) from 
all defined product scopes. This would leverage 
'IEnvironmentSetupParticipant.environment_needs_upgrade' and 
'IEnvironmentSetupParticipant.upgrade_environment', executing those two 
from within all product scopes (+global one).
In case of the aforementioned component, the component would, in 
'environment_needs_upgrade', normally do a 'SELECT' on 'system' table. 
As it would be running from within product scope, it would get version 
for that specific product. The view of the database would be scoped to 
that specific product so any update, alter or whatever database 
operation it would perform would be limited to that specific scope. 
'INSERT's or 'UPDATE's into 'system' table during the upgrade would also 
behave as it is the case in product addition, it'd be limited to the 
specific product's scope.
Note that upgrades for all products could be run from within the same 
database transaction. The 'needs_upgrade' and 'upgrade' methods of 
'trac.env.Environment' should be overriden to provide product aware 
upgrade process.

-- Multi-product aware component
Multi-product aware component's 'IGlobalSetupParticipant' is only 
invoked once, in global environment scope.

This is how, in my understanding, things should be. I'm not sure there's 
really any other way of implementing this ...


View raw message