airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Candler (JIRA)" <>
Subject [jira] [Created] (AIRFLOW-200) Hiding import errors / missing dependency on unicodecsv
Date Wed, 01 Jun 2016 07:46:59 GMT
Brian Candler created AIRFLOW-200:

             Summary: Hiding import errors / missing dependency on unicodecsv
                 Key: AIRFLOW-200
             Project: Apache Airflow
          Issue Type: Bug
          Components: core
    Affects Versions: Airflow
         Environment: ubuntu 14.04 (python 2.7), new virtualenv, pip install airflow
            Reporter: Brian Candler
            Priority: Minor

When running the quickstart instructions at inside
a clean virtualenv:

ERROR [airflow.models.DagBag] Failed to import:
Traceback (most recent call last):
line 247, in process_file
    m = imp.load_source(mod_name, filepath)
line 26, in <module>
    from airflow.operators import BashOperator, HiveOperator,
ImportError: cannot import name HiveOperator

Unfortunately that message doesn't help diagnose the problem, which is being hidden by auto-import

It requires manually probing imports from the true source modules:

>>> from airflow.operators.hive_operator import HiveOperator
ImportError: cannot import name HiveCliHook

>>> from airflow.hooks.hive_hooks import HiveCliHook
ImportError: No module named unicodecsv

Aha. "pip install unicodecsv" fixes the error.

So I'd suggest two issues:

1. Add a packaging dependency on unicodecsv to fix this particular problem

2. Fix the auto-import magic so that it doesn't suppress these errors

This message was sent by Atlassian JIRA

View raw message