airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Napolitano <>
Subject Re: Suggested way of passing "input parameters" to a DAG run?
Date Wed, 03 Aug 2016 22:42:03 GMT
What I can say is that we use it a lot, but very lightly.  We basically use
it to communicate the S3 key for a flat file between operators.

Definitely don't use it to send actual data :)


On Wed, Aug 3, 2016 at 6:21 PM, Andrew Phillips <> wrote:

> Let me know if that helps, or if I completely misunderstood :)
> That helps, indeed - thanks, Joe! We were in fact going down exactly this
> path as an alternative; we were just a bit hesitant to use XComs based on
> the following comment in the docs [1]:
> "If it absolutely can’t be avoided, Airflow does have a feature for
> operator cross-communication called XCom that is described elsewhere in
> this document."
> The statements talks about sharing information *between* tasks, but we
> weren't sure if this should be read as "stay away from XComs unless there's
> no other option". Curious to hear the community's thoughts on that.
> Thanks for the quick response!
> ap
> [1]

*Joe Napolitano *| Sr. Data Engineer | 5 Crosby Street, New York, NY 10013

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message