brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Svetoslav Neykov <svetoslav.ney...@cloudsoftcorp.com>
Subject Re: [PROPOSAL] Controlling effectors concurrency
Date Wed, 11 Jan 2017 15:55:00 GMT
Alex,

I like the task based approach and agree it's something we need to push for. I'm not convinced
that locking-like behaviour should be represented as tasks though. It brings procedural style
flavour to concurrency - for example in your original example there's a separate tasks for
acquiring and releasing the lock. They could easily get misplaced, overwritten, forgotten,
etc. Not clear how releasing it works in case of errors.
What I'd prefer is more declarative approach where the tasks are grouped and the locking requirements
are applied on the group. At worst referencing a start task and optionally an end task (with
the default of the parent task existing).

Plugging into the lifecycle of entities has other use-cases. No matter how an entity is defined
- whether using the current monolithic START effector or one composited of smaller tasks -
there's no way to be notified when its tasks get executed. Some concrete examples - the "SystemServiceEnricher"
which looks for the "launch" task and can be applied on any "SoftwareProcessEnity"; An entity
which needs to do some cleanup based on the shutdown of another entity (DNS blueprints); latching
at arbitrary points during entity lifecycle; etc.

One of the alternatives I mention in the original email (Effector execution notifications)
is a step in the "task based" approach direction. It's still in Java, but we are splitting
the monolith effectors into smaller building blocks, which could in future get reused in YAML.

So to summarise - +1 for tasks as building blocks, but still need visibility into the executing
blocks.

> PS - as a near-term option if needed we could extend SoftwareProcess LATCH to do something
special if the config/sensor it is given is a "Semaphore" type

What do you think the behaviour should be here - releasing the semaphore after the corresponding
step completes or at the end of the wrapping effector? I think this is defined by the blueprint
authors. And making this configurable adds even more complexity. Instead could invest in developing
the above functionality.

Svet.



> On 11.01.2017 г., at 16:25, Alex Heneveld <alex.heneveld@cloudsoftcorp.com> wrote:
> 
> 
> svet, all,
> 
> lots of good points.
> 
> the idea of "lifecycle phases" in our software process entities has grown to be something
of a monster, in my opinion.  they started as a small set of conventions but they've grown
to the point where it's the primary hook for yaml and people are wanting "pre.pre.install".
> 
> it's (a) misguided since for any value of N, N phases is too few and (b) opaque coming
from a java superclass.
> 
> a lot of things rely on the current SoftwareProcess so not saying we kill it, but for
the community it would be much healthier (in terms of consumers) to allow multiple strategies
and especially focus on *the reusability of tasks in YAML* -- eg "provision" or "install template
files" or "create run dir" -- so people can write rich blueprints that don't require magic
lifecycle phases from superclasses.
> 
> the "init.d" numbered approach svet cites is one way these are wired together, with extensions
(child types) able to insert new numbered steps or override with the number and label.  but
that would be one strategy, there might be simpler ones that are just a list, or other sequencing
strategies like precondition/action/postcondition or runs-before/runs-after (where to extend
something, you can say `my-pre-customize: { do: something, run-before: launch, run-after:
customize }`).
> 
> we're not sure exactly how that would look but wrt mutex logic the idea that adjuncts
plug into the entity lifecycles feels wrong, like it's pushing for more standardisation in
lifecycle phases where things can plug in.   whereas with a task approach we can have effector
definitions being explicit about synchronization, which i think is better.  and if they want
to make that configurable/pluggable they can do this and be explicit about how that is done
(for instance it might take a config param, even a config param (or child or relation) which
is an entity and call an effector on that if set).
> 
> concretely in the examples i'm saying instead of ENRICHER APPROACH
> 
>        memberSpec:
>          $brooklyn:entitySpec:
>            type: cluster-member
>            brooklyn.enrichers:
>            - type: org.apache.brooklyn.enricher.stock.AquirePermissionToProceed
>              brooklyn.config:
>                stage: post.provisioning
>                # lifecycle stage "post.provisioning" is part of cluster-member and
>                # the enricher above understands those stages
> 
> we concentrate on a way to define tasks and extend them, so that we could instead have
a TASK APPROACH:
> 
>        memberSpec:
>          $brooklyn:entitySpec:
>            type: cluster-member
>            effectors:
>              start:
>                035-pre-launch-get-semaphore: { acquire-semaphore: ... }
>                # assume 040-launch is defined in the parent "start" yaml defn
>                # using a new hypothetical "initd" yaml-friendly task factory,
>                # acquire-semaphore is a straightforward task;
>                # scope can come from parent/ancestor task,
>                # also wanted to ensure mutex is not kept on errors
> 
> or
> 
>        memberSpec:
>          $brooklyn:entitySpec:
>            type: cluster-member
>            effectors:
>              start:
>                my-pre-launch:
>                  task:  { acquire-semaphore: ... }
>                  run-before: launch
>                  run-after: customize
>                  # launch and customize defined in the parent "start" yaml defn,
>                  # using a new hypothetical "ordered-labels" yaml-friendly task factory;
>                  # acquire-semaphore and scope are as in the "initd" example
> 
> 
> both approaches need a little bit of time to get your head around the new concepts but
the latter approach is much more powerful and general.  there's a lot TBD but the point i'm
making is that *if we make tasks easier to work with in yaml, it becomes more natural to express
concurrency control as tasks*.
> 
> 
> PS - as a near-term option if needed we could extend SoftwareProcess LATCH to do something
special if the config/sensor it is given is a "Semaphore" type
> 
> best
> alex
> 


Mime
View raw message