airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lewis John McGibbney <lewi...@apache.org>
Subject Re: Guidance/best practice for mocking task instance output for unit testing DAGs
Date Wed, 04 Aug 2021 14:26:02 GMT
I will add that the unit tests we are writing relate to DAGs which are written using the Taskflow
API.
We have come to understand that XCom cannot be mocked in unit tests... and I think that this
is the source of our problem. Therefore we have experimented with manually defining XCom pull
and push constructs as follows

    pulled_value = ti.xcom_pull(key='someKey', task_ids='push_to_xcoms')

Is this the correct way to write task instance unit tests which require input parameters from
XCom?

Thanks
lewismc

On 2021/07/29 16:39:02, Lewis John McGibbney <lewismc@apache.org> wrote: 
> Hi users@,
> Say I want to unit test Task A which accepts one or more parameters. These parameters
are the outputs of some other task which I want to mock. Can someone provide a code example
of how I would do that?
> We are using pytest as opposed to unittest but an example of either would be greatly
appreciated.
> I studied several articles [0] [1] [2] which were informative but I think lack the content
which explains what I want to do above.
> I did find the following code example [3] which I feel is on the correct track but I
am not confident.
> Thanks for any consideration.
> lewismc
> 
> [0] https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#testing-a-dag
> [1] https://blog.usejournal.com/testing-in-airflow-part-1-dag-validation-tests-dag-definition-tests-and-unit-tests-2aa94970570c
> [2] https://medium.com/@chandukavar/testing-in-airflow-part-2-integration-tests-and-end-to-end-pipeline-tests-af0555cd1a82
> [3] https://github.com/apache/airflow/blob/main/tests/ti_deps/deps/test_dagrun_exists_dep.py
> 

Mime
View raw message