airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Potiuk <Jarek.Pot...@polidea.com>
Subject Re: Dynamic word in principles is misleading
Date Wed, 16 Sep 2020 12:52:57 GMT
Just a few comments:
- the structure can change dynamically. Just "slowly" (I know it's not very
precise). They will work fine when the structure change "slowly" and the UI
will also not be fully reflecting the reality while transition is happening
but it will work.
- we are working on becoming "true" to the dynamic promise - we have a work
in progress about DAG versioning
https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-36+DAG+Versioning
and once implemented, the dynamic nature that was promised but not yet
delivered. I'd really love if you join the discussions there and maybe even
contribute to making it happen :)

J.

On Wed, Sep 16, 2020 at 9:30 AM Johnny Bravo <flappy.forever@gmail.com>
wrote:

> Hi!
>
> I'm a user of your great tool. I have already created few DAGs and
> some of them are even quite complex (in terms of carefully crafted
> custom operators as well as in terms of size and complexity of
> dependencies between tasks). Before getting in that place I've gone a
> long way for understanding the semantics used by Airflow.
>
> Recently I have entered again to the Airflow Apache page and to my
> surprise found that actually the part on which I spent most of time
> during solution development is somehow wrongly described on the
> Airflow page. It is all about the word "dynamic" on your principles
> list.
>
> Airflow DAGs are not dynamic. To shorten discussion, it should be
> called "Programmable", which means that we can craft DAG with the use
> of algorithms or external knowledge, but final shape cannot change
> during execution as the DAG must be fully known before running so
> actually these are static.
>
> Lets now find out meaning of two words:
>
> dynamic - "(of a process or system) characterized by constant change,
> activity, or progress." (afer google translator) and "In general,
> dynamic means energetic, capable of action and/or change, or forceful,
> while static means stationary or fixed. In computer terminology,
> dynamic usually means capable of action and/or change, while static
> means fixed." (after techtarget.com).
>
>
>
> programmable - (of a computer or other machine) able to be provided
> with coded instructions for the automatic performance of a task (after
> google translator) and here Computer_programming @ wikipedia.org
>
> So for the sake of stopping giving newbies' heads headaches, stop
> using the word "dynamic" in the current context.
>
> Change my mind ;-)
>
> Cheers,
> JB.
>


-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>

Mime
View raw message