airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremiah Lowin <jlo...@apache.org>
Subject Re: Xcom related security issue
Date Sun, 19 Feb 2017 17:37:28 GMT
Rui,

Thanks for pointing this out, it's a valid concern.

I personally have no issue with swapping Pickle -> JSON, but there may be
many Airflow users relying on the current behavior and I don't want to
invalidate their DAGs with a PR.

On the other hand, I'm not sure of a way to "gently" deprecate the
PickleType. Perhaps step 1 is to check if an XCom can be JSON serialized
and if it can't, print a warning? Then step 2 is to enforce JSON
serialization at a future date.

Any suggestions of how to implement this?

J

On Sat, Feb 18, 2017 at 10:16 AM Rui Wang <rui.wang@airbnb.com.invalid>
wrote:

> Hi,
>
> I created an JIRA issue: https://issues.apache.org/jira/browse/AIRFLOW-855
> .
>
>
> The JIRA task above gives pretty rich context. Briefly speaking, PickleType
> gives the possible that run code/command on remote machines. This type can
> serialize objects, which is a wide scope. I am wondering what kind of use
> cases you have for using Xcom and its PickleType. If the use cases show the
> possibility that replacing PickleType with JSON type, the probably this
> security issue can be solved by using JSON type instead,
>
>
> Thanks,
> Rui Wang
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message