airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: Broken unit tests -- request for help
Date Tue, 09 Aug 2016 13:15:38 GMT
This is fixed. This was due to an issue with Travis’ cache: it can corrupt the cache and
therefore
untarring fails. Now hive is re-downloaded in case this happens.

- Bolke

> Op 4 aug. 2016, om 14:50 heeft Jeremiah Lowin <jlowin@apache.org> het volgende
geschreven:
> 
> We have a few non-deterministic unit test failures that are affecting many
> -- but not all -- PRs. I believe they are being ignored as "unrelated" but
> they have the potential to mask real issues and should be addressed.
> Unfortunately they're out of my expertise so I'm going to list the ones
> I've identified and hope someone smarter than me can see if they can help!
> 
> In particular, we have a number of simple PR's that should obviously have
> no problems (typos, readme edits, etc.) that are nonetheless failing tests,
> causing frustration for all. Here is one from just this morning:
> https://github.com/apache/incubator-airflow/pull/1705/files
> 
> Thanks in advance!
> 
> 1. Python 3 Mysql (this one is pretty common), due to not being able to
> find "beeline" which I believe is related to Hive. This is the error:
> 
> ======================================================================
> 
> ERROR: test_mysql_to_hive_partition (tests.TransferTests)
> 
> ----------------------------------------------------------------------
> 
> Traceback (most recent call last):
> 
>  File "/home/travis/build/apache/incubator-airflow/tests/operators/operators.py",
> line 208, in test_mysql_to_hive_partition
> 
>    t.run(start_date=DEFAULT_DATE, end_date=DEFAULT_DATE, force=True)
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/models.py",
> line 2350, in run
> 
>    force=force,)
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py",
> line 54, in wrapper
> 
>    result = func(*args, **kwargs)
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/models.py",
> line 1388, in run
> 
>    result = task_copy.execute(context=context)
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/operators/mysql_to_hive.py",
> line 131, in execute
> 
>    recreate=self.recreate)
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/hooks/hive_hooks.py",
> line 322, in load_file
> 
>    self.run_cli(hql)
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/hooks/hive_hooks.py",
> line 212, in run_cli
> 
>    cwd=tmp_dir)
> 
>  File "/opt/python/3.4.2/lib/python3.4/subprocess.py", line 858, in __init__
> 
>    restore_signals, start_new_session)
> 
>  File "/opt/python/3.4.2/lib/python3.4/subprocess.py", line 1456, in
> _execute_child
> 
>    raise child_exception_type(errno_num, err_msg)
> 
> nose.proxy.FileNotFoundError: [Errno 2] No such file or directory: 'beeline'
> 
> 
> 2. Python 3 Postgres (this one is really infrequent):
> 
> ======================================================================
> 
> FAIL: Test that ignore_first_depends_on_past doesn't affect results
> 
> ----------------------------------------------------------------------
> 
> Traceback (most recent call last):
> 
>  File "/home/travis/build/apache/incubator-airflow/tests/jobs.py",
> line 349, in test_dagrun_deadlock_ignore_depends_on_past
> 
>    run_kwargs=dict(ignore_first_depends_on_past=True))
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py",
> line 54, in wrapper
> 
>    result = func(*args, **kwargs)
> 
>  File "/home/travis/build/apache/incubator-airflow/tests/jobs.py",
> line 221, in evaluate_dagrun
> 
>    self.assertEqual(ti.state, expected_state)
> 
> nose.proxy.AssertionError: None != 'success'
> 
> 3. Mysql (py2 and py3, infrequent). This appears to happen when the
> SLA code is called wiht mysql. Bizarrely, this doesn't appear to
> actually raise an error in the test -- it just prints a logging error.
> It must be trapped somewhere.
> 
> ERROR [airflow.jobs.SchedulerJob] Boolean value of this clause is not defined
> 
> Traceback (most recent call last):
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/jobs.py",
> line 667, in _do_dags
> 
>    self.manage_slas(dag)
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py",
> line 53, in wrapper
> 
>    result = func(*args, **kwargs)
> 
>  File "/home/travis/build/apache/incubator-airflow/airflow/jobs.py",
> line 301, in manage_slas
> 
>    .all()
> 
>  File "/home/travis/build/apache/incubator-airflow/.tox/py34-cdh-airflow_backend_mysql/lib/python3.4/site-packages/sqlalchemy/sql/elements.py",
> line 2760, in __bool__
> 
>    raise TypeError("Boolean value of this clause is not defined")
> 
> TypeError: Boolean value of this clause is not defined


Mime
View raw message