incubator-bloodhound-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olemis Lang <ole...@gmail.com>
Subject Re: [Apache Bloodhound] #404: Populate default schema on product addition
Date Tue, 26 Feb 2013 09:01:14 GMT
On 2/25/13, Jure Zitnik <jure@digiverse.si> wrote:
> On 2/24/13 7:07 AM, Olemis Lang wrote:
>> On 2/22/13, Jure Zitnik <jure@digiverse.si> wrote:
[....]
>
> Let's take one step back and take a look at the following use-case.
>

ok, I'll follow from this point forward

> Let's assume we have a component that is a setup participant. Let's say
> that non multi-product aware components are setup participants when they
> implement 'IEnvironmentSetupParticipant' interface and that
> multi-product aware components are setup participants when they
> implement 'IGlobalSetupParticipant'. Let's also assume (for now) that
> the 'IEnvironmentSetupParticipant' and 'IGlobalSetupParticipant'
> interface's signatures are the same.
>

ok ... we can handle that

> Let's say that the 'environment_created' method of the component does
> the following:
> 1. creates table named 'my_table'
> 2. inserts into 'my_table' data that aggregates/references some data
> from system tables ('ticket', 'component',...)
> 3. creates a file in 'env.path' that holds some environment specific data
> 4. inserts it's database version into 'system' table
>
>
> Now, let's discuss the relevant scenarios: (global) environment creation
> (install from scratch), product addition and environment upgrade.

There's an important case : upgrade with installed plugins and
preexisting data . maybe covered by the third one . It involves data
migration and a few other corner cases ... but your initial scenarios
are just find to get this sorted out .

> I'll
> describe scenarios for both cases, first for non multi-product aware
> component and then for multi-product aware component.
>

I'll follow this sequence with the purpose in mind of finding a way to
work around the current design limitation of having O(p) plugin tables
.

So I'm basically thinking of having one of the following DB schemas :

  1. A single table scoping all plugin resources at product level
  2. Two tables : one unchanged for products in global scope ,
      the second one for product resources .

I'll only care about DB stuff because that's actually what really
matters and the cause of this bottleneck . I'll also ignore MP-aware
plugins / components given the fact that there is no such thing yet
and we will be able to do whatever we consider necessary to make that
work , as opposite to legacy plugins .

> 1. Global environment creation
> -- Non multi-product aware component
> In this case, the components
> 'IEnvironmetSetupParticipant.environment_created' should be executed in
> global environment only (as there are no products). Result of this would
> be 'my_table' table with aggregated/referenced data from 'null' product
> scope (global scope). Note that the translator does not prefix table
> names when product is 'null'. File that the component creates would be
> stored in the global environment path, the 'system' table would get a
> record with 'null' product prefix.
>
> -- Multi-product aware component
> Same as in case of non multi-product aware component, everything happens
> in global environment and through (new) 'IGlobalSetupParticipant'
> interface. Results of 'environment_created' (tables, files, etc) are the
> same as in the case of non multi-product aware component.
>

Looking at this with the previous goals in mind , IMO at this point we
should be doing this (numbers will match scenarios listed above)

  1. create plugin table with extra 'product' column for prefix
      and perform all other outstanding DDL and DML statements
      based on this modified schema .
  2. create plugin table with extra 'product' prefix column

In any case the goal is to get a fully functional schema for plugins
comprising no more than two tables . That would be everyhting needed
to make all imaginable plugins work with activated plugin version .

The fact that components in product environments MUST also be enabled
in global env implies that a component class in product scope will
never be instantiated before its global peer is queried for upgrades .

> 2. Product (with 'MYPRODUCT' prefix) addition
> -- Non multi-product aware component
> Adding a product should invoke component's
> 'IEnvironmentSetupParticipant.environment_created' method

Ok . We agree up to this point , afaict since the beginning .

> from within
> 'ProductEnvironment('MYPRODUCT')'.

Nope . I disagree here . IMO , this should be happeing immediately
after creating the product either in MP system's CRUD handler , a
dedicated implementation of a product resource listener or a
combination of both .

> This would enable SQL translator in
> that scope with the known rules. Result of this would be
> 'MYPRODUCT_my_table' with aggregated/referenced data from 'MYPRODUCT's
> scope. File would be stored in 'MYPRODUCT' product's path. 'system'
> table would get a record with 'MYPRODUCT' product prefix.
>

In both scenarios I mentioned above (1) and (2) we'll need the
translators , that's a fact.

Nevertheless notice that global upgrade would shape the schema for any
product scope as well . So my idea is to shut off DDL statements at
this point . OTOH DML statements will remain translated considering
product prefix as data still needs to be inserted / updated / selected
/ removed .

> -- Multi-product aware component
> Multi-product aware component is notified of product addition through
> IResourceChangeListener.
>

+1
JFTR ... one such listener component (or equivalent) should be dealing
with the process mentioned above for non multi-product aware setup
participants

> 3. Environment upgrade
> -- Non multi-product aware component
> Following the normal process (in 'trac.env.Environment'), each non
> multi-product setup participant should be upgraded (if required) from
> all defined product scopes. This would leverage
> 'IEnvironmentSetupParticipant.environment_needs_upgrade' and
> 'IEnvironmentSetupParticipant.upgrade_environment', executing those two
> from within all product scopes (+global one).

JFTR : this is part of what I referred before as «replay» as I was
actually thinking of an upgrade with products in the DB , preexisting
plugin data and pending upgrades applied upon products in a batch
while still inside the global upgrade loop . IMO this should happen
inside the global upgrade loop .

> In case of the aforementioned component, the component would, in
> 'environment_needs_upgrade', normally do a 'SELECT' on 'system' table.
> As it would be running from within product scope, it would get version
> for that specific product. The view of the database would be scoped to
> that specific product so any update, alter or whatever database
> operation it would perform would be limited to that specific scope.
> 'INSERT's or 'UPDATE's into 'system' table during the upgrade would also
> behave as it is the case in product addition, it'd be limited to the
> specific product's scope.

With my suggestions , if feasible , all this will not be necessary due
to the fact that there will be no more than two tabes for this .

> Note that upgrades for all products could be run from within the same
> database transaction. The 'needs_upgrade' and 'upgrade' methods of
> 'trac.env.Environment' should be overriden to provide product aware
> upgrade process.
>

No need to do so if we just make the MP system responsible for
replaying product upgrades . Hope you understand what I mean now , and
why I advocate for doing so in the global upgrade loop . Product
specific transactions , behaving like contexts , will be nested within
the global upgrade transaction context as well .

> -- Multi-product aware component
> Multi-product aware component's 'IGlobalSetupParticipant' is only
> invoked once, in global environment scope.
>

+1

>
> This is how, in my understanding, things should be. I'm not sure there's
> really any other way of implementing this ...
>

We shall see what you all think about my proposed hack . I look
forward to read further comments .

-- 
Regards,

Olemis.

Mime
View raw message