airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: Pulling XCom from parent DAG in a sub DAG template
Date Thu, 02 Feb 2017 07:58:16 GMT
I think `dag.parent_dag.dag_id` should work, either in a template of
through the context in a PythonOperator.

Max

On Wed, Feb 1, 2017 at 4:23 PM, Daniel Huang <dxhuang@gmail.com> wrote:

> Hey everyone,
> I currently have a parent DAG that generates a filename based on the
> current timestamp. That same filename needs to be used down the line in
> multiple operators, including some subdag operators. So I was hoping to use
> XComs to pass this filename along. It works fine for operators within the
> parent DAG, but not for any operators in the sub DAGs because the dag_id
> arg on xcom_pull() defaults to self.dag_id. I'm pulling this XCom value
> from templates, so I don't want to have to hardcode a parent's dag id in it
> because it should work for different parent dags. The sub dag template
> would still have to make assumptions about what is available in xcom under
> a task id, but I think that's alright?
>
> So my question is if there is a way to programmatically specify the parent
> dag id in an xcom pull from within a template or if there's a better way of
> doing what I'm trying to achieve. Note, I also have sub dags within sub
> dags that require the same filename.
>
> If there is no way to this already, I did come across
> https://issues.apache.org/jira/browse/AIRFLOW-54 and wondered if we need
> something similar for the dag_id arg, like ti.xcom_pull(dag_id='@parent',
> task_ids='foo').
>
> Thanks,
> Daniel
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message