airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Beauchemin <maximebeauche...@gmail.com>
Subject Re: Broken unit tests -- request for help
Date Tue, 09 Aug 2016 22:59:17 GMT
Nice, thanks for fixing this!

Were you able to clear the cache? Seems like I can't terminate builds since
I don't have admin rights on the underlying repo, so I was assuming I
couldn't clear the cache as welll...

Max

On Tue, Aug 9, 2016 at 6:15 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> This is fixed. This was due to an issue with Travis’ cache: it can corrupt
> the cache and therefore
> untarring fails. Now hive is re-downloaded in case this happens.
>
> - Bolke
>
> > Op 4 aug. 2016, om 14:50 heeft Jeremiah Lowin <jlowin@apache.org> het
> volgende geschreven:
> >
> > We have a few non-deterministic unit test failures that are affecting
> many
> > -- but not all -- PRs. I believe they are being ignored as "unrelated"
> but
> > they have the potential to mask real issues and should be addressed.
> > Unfortunately they're out of my expertise so I'm going to list the ones
> > I've identified and hope someone smarter than me can see if they can
> help!
> >
> > In particular, we have a number of simple PR's that should obviously have
> > no problems (typos, readme edits, etc.) that are nonetheless failing
> tests,
> > causing frustration for all. Here is one from just this morning:
> > https://github.com/apache/incubator-airflow/pull/1705/files
> >
> > Thanks in advance!
> >
> > 1. Python 3 Mysql (this one is pretty common), due to not being able to
> > find "beeline" which I believe is related to Hive. This is the error:
> >
> > ======================================================================
> >
> > ERROR: test_mysql_to_hive_partition (tests.TransferTests)
> >
> > ----------------------------------------------------------------------
> >
> > Traceback (most recent call last):
> >
> >  File "/home/travis/build/apache/incubator-airflow/tests/
> operators/operators.py",
> > line 208, in test_mysql_to_hive_partition
> >
> >    t.run(start_date=DEFAULT_DATE, end_date=DEFAULT_DATE, force=True)
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/models.py",
> > line 2350, in run
> >
> >    force=force,)
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py",
> > line 54, in wrapper
> >
> >    result = func(*args, **kwargs)
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/models.py",
> > line 1388, in run
> >
> >    result = task_copy.execute(context=context)
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/
> operators/mysql_to_hive.py",
> > line 131, in execute
> >
> >    recreate=self.recreate)
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/
> hooks/hive_hooks.py",
> > line 322, in load_file
> >
> >    self.run_cli(hql)
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/
> hooks/hive_hooks.py",
> > line 212, in run_cli
> >
> >    cwd=tmp_dir)
> >
> >  File "/opt/python/3.4.2/lib/python3.4/subprocess.py", line 858, in
> __init__
> >
> >    restore_signals, start_new_session)
> >
> >  File "/opt/python/3.4.2/lib/python3.4/subprocess.py", line 1456, in
> > _execute_child
> >
> >    raise child_exception_type(errno_num, err_msg)
> >
> > nose.proxy.FileNotFoundError: [Errno 2] No such file or directory:
> 'beeline'
> >
> >
> > 2. Python 3 Postgres (this one is really infrequent):
> >
> > ======================================================================
> >
> > FAIL: Test that ignore_first_depends_on_past doesn't affect results
> >
> > ----------------------------------------------------------------------
> >
> > Traceback (most recent call last):
> >
> >  File "/home/travis/build/apache/incubator-airflow/tests/jobs.py",
> > line 349, in test_dagrun_deadlock_ignore_depends_on_past
> >
> >    run_kwargs=dict(ignore_first_depends_on_past=True))
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py",
> > line 54, in wrapper
> >
> >    result = func(*args, **kwargs)
> >
> >  File "/home/travis/build/apache/incubator-airflow/tests/jobs.py",
> > line 221, in evaluate_dagrun
> >
> >    self.assertEqual(ti.state, expected_state)
> >
> > nose.proxy.AssertionError: None != 'success'
> >
> > 3. Mysql (py2 and py3, infrequent). This appears to happen when the
> > SLA code is called wiht mysql. Bizarrely, this doesn't appear to
> > actually raise an error in the test -- it just prints a logging error.
> > It must be trapped somewhere.
> >
> > ERROR [airflow.jobs.SchedulerJob] Boolean value of this clause is not
> defined
> >
> > Traceback (most recent call last):
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/jobs.py",
> > line 667, in _do_dags
> >
> >    self.manage_slas(dag)
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py",
> > line 53, in wrapper
> >
> >    result = func(*args, **kwargs)
> >
> >  File "/home/travis/build/apache/incubator-airflow/airflow/jobs.py",
> > line 301, in manage_slas
> >
> >    .all()
> >
> >  File "/home/travis/build/apache/incubator-airflow/.tox/py34-
> cdh-airflow_backend_mysql/lib/python3.4/site-packages/
> sqlalchemy/sql/elements.py",
> > line 2760, in __bool__
> >
> >    raise TypeError("Boolean value of this clause is not defined")
> >
> > TypeError: Boolean value of this clause is not defined
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message