gump-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leo Simons <m...@leosimons.com>
Subject Re: Gump3 inter-component orchestration/communications
Date Tue, 12 Apr 2005 15:36:45 GMT
On 11-04-2005 01:19, "Adam  Jack" <ajack@mric.coop> wrote:
> Why are CvsUpdater, SvnUpdater pre-process plug-ins, not 'process' visitors?
> Clearly (as of now) it make little difference, I'm just trying to
> understand.

I don't know. It's a good question!

> I'm not sure I'm comfortable with the three pre-process/process/post-process
> walks. Is this a Gump2 hold over that we want in Gump3?

Well, what I tried to do so far is just make it easier to change these
things. Right now, making CvsUpdate part of "process" instead of
"pre-process" is something like a three-line change to the config.py file.
Beyond that, I don't have good answers to many of these kinds of questions,
other than "its a hunch".

> I'm not wanting us
> to design a generic mechanism, just solve our problem, but is the approach
> the right one? Why are these three walks better than (say) putting
> Updates/Builders/Databasers(DynaGumper) in sequence inside one mutlicast
> plugin?

Do you think we could do without multiple stages completely?

I don't remember exactly why I put in 3 stages, but I think we discussed it
a little before. Maybe indeed simply because gump2 has 3 stages, I didn't
see much wrong with it, so I kept it that way. If it ain't broke don't fix
it.

Here's some test driven design for you: we don't have a test case that's
going to fail if we make it into one stage only, so we probably only need
one :-D

> One side effect of the current approach is that (during a long slow build)
> we'll only update the database at the end of the run, after all modules have
> been updated and all projects have been built. This was something we found
> unpleasant w/ Gump2, 'cos folks want more instant gratification.

Really? I've always found it really annoying to want to debug something and
then seeing that a build is "in progress" meaning part of that tree is from
the run before and part of it is new. Very confusing...

> We ought
> update a DB and (say) notify of a failure (e.g. e-mail or RSS) ASAP [i.e. at
> t he time of the processing], to give communities the most opportunity to
> solve things before the next Gump run.

Like before, I agree with you here, but it makes a lot of stuff more
complex. For example, its difficult to report on all the side effects of a
failure if those side effects haven't actually occurred yet. You shouldn't
tell project A that project B has failed because it failed without providing
a link detailing the failure of project B showing how B depends on A.

Anyway, you can have a database plugin at the end of the pre-process stage
that puts info into the database about the results of pre-processing, then
another database plugin at the end of the processing stage about the results
of the processing, then another database plugin at the end of the
post-processing stage that handles all the "linking" bits as in the example
above. And you can move plugins from one stage to another, or "soft-merge"
them using a multicaster or something similar.

Anyway, maybe that's why I put in three distinct stages. You can define lots
of simple rules such as

"after pre-processing is done, projects that don't have the nobuild property
set are guaranteed to be current"

"post-processing plugins can access all model information and all context
information added by all pre-processing and processing plugins"

"plugins in different stages should interact exclusively through the model"

"post-processing plugins should not interact with each other"

"post-processing plugins should not modify the model"

"pre-processing nor processing plugins should query the database"

Which are tied to a particular stage and are more important conceptually
than implementation-wise. That makes plugin writing easier. Of course, those
rules need to be defined, and we need to figure out which ones make sense.
Not even talking about testing them or gently guiding plugin writers to use
them. See how complex a real simple plugin model already is?

> I like how Gump3 is simple, and fires events out to plugins (letting them
> decide if they wish to process them, not centralizing this logic) however I
> think that'll put a higher burden onto inter-component orchestration and
> communications. I think using the model object as a "message board"
                                                      ^^^
That's a cool way to put it. Lot better than referring to things like
"inter-process communication" or other ugly stuff ;)

> by
> writing attributes is the right Pythonic way to solve the latter. It is the
> former that worries me.

Here's a hunch: provided plugins are small and most model-independent
functionality ends up in isolated helper classes/methods, I believe we don't
need much "inter-component orchestration" at all. Or in TDD speak: gimme a
test case that shows we do :-D

> I don't have a solution here, I'm just asking questions...

And they're probably all the right questions! I don't have the answers
either. However, I'm confident that the current setup is flexible enough to
experiment with different answers to these questions. We can just try them
out, having requirements and plugins drop out of our use cases and unit
tests.

It's a real fun way of doing this imo: always just go ahead and make change
x, *provided* it results in

A) simpler code
   - and/or -
B) fewer failing unit tests

And the reverse, if someone disagrees with change x they should either
provide even simpler code or a failing unit test.


Phew, long e-mail! You make me think hard... :-D

Cheers,

Leo



---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@gump.apache.org
For additional commands, e-mail: general-help@gump.apache.org


Mime
View raw message