airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lulu Cheng (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-1539) python3 error import Dag
Date Mon, 28 Aug 2017 22:12:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lulu Cheng updated AIRFLOW-1539:
--------------------------------
    Description: 
Seeing the following error when using python3 (fine with python2)

{code:python}
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/airflow/models.py", line 263, in process_file
    m = imp.load_source(mod_name, filepath)
  File "/opt/conda/lib/python3.6/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 675, in _load
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/root/airflow/dags/LoadS3ToHiveTest.py", line 2, in <module>
    from LoadS3ToHive import default_args, download_s3_to_hdfs_templated_command, generate_download_s3_to_hdfs_task,
\
  File "/root/airflow/dags/LoadS3ToHive.py", line 74, in <module>
    globals()[task_name] = DAG(task_name, default_args=default_args, schedule_interval=None)
  File "/opt/conda/lib/python3.6/site-packages/airflow/models.py", line 2664, in __init__
    self.fileloc = inspect.getsourcefile(inspect.stack()[1][0])
  File "/opt/conda/lib/python3.6/inspect.py", line 1465, in stack
    return getouterframes(sys._getframe(1), context)
  File "/opt/conda/lib/python3.6/inspect.py", line 1442, in getouterframes
    frameinfo = (frame,) + getframeinfo(frame, context)
  File "/opt/conda/lib/python3.6/inspect.py", line 1411, in getframeinfo
    filename = getsourcefile(frame) or getfile(frame)
  File "/opt/conda/lib/python3.6/inspect.py", line 666, in getsourcefile
    if getattr(getmodule(object, filename), '__loader__', None) is not None:
  File "/opt/conda/lib/python3.6/inspect.py", line 703, in getmodule
    if ismodule(module) and hasattr(module, '__file__'):
  File "/opt/conda/lib/python3.6/site-packages/airflow/utils/timeout.py", line 38, in handle_timeout
    raise AirflowTaskTimeout(self.error_message)
airflow.exceptions.AirflowTaskTimeout: Timeout
{code}


  was:
Seeing the following error when using python3 (fine with python2)
{{Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/airflow/models.py", line 263, in process_file
    m = imp.load_source(mod_name, filepath)
  File "/opt/conda/lib/python3.6/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 675, in _load
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
  File "/root/airflow/dags/LoadS3ToHiveTest.py", line 2, in <module>
    from LoadS3ToHive import default_args, download_s3_to_hdfs_templated_command, generate_download_s3_to_hdfs_task,
\
  File "/root/airflow/dags/LoadS3ToHive.py", line 74, in <module>
    globals()[task_name] = DAG(task_name, default_args=default_args, schedule_interval=None)
  File "/opt/conda/lib/python3.6/site-packages/airflow/models.py", line 2664, in __init__
    self.fileloc = inspect.getsourcefile(inspect.stack()[1][0])
  File "/opt/conda/lib/python3.6/inspect.py", line 1465, in stack
    return getouterframes(sys._getframe(1), context)
  File "/opt/conda/lib/python3.6/inspect.py", line 1442, in getouterframes
    frameinfo = (frame,) + getframeinfo(frame, context)
  File "/opt/conda/lib/python3.6/inspect.py", line 1411, in getframeinfo
    filename = getsourcefile(frame) or getfile(frame)
  File "/opt/conda/lib/python3.6/inspect.py", line 666, in getsourcefile
    if getattr(getmodule(object, filename), '__loader__', None) is not None:
  File "/opt/conda/lib/python3.6/inspect.py", line 703, in getmodule
    if ismodule(module) and hasattr(module, '__file__'):
  File "/opt/conda/lib/python3.6/site-packages/airflow/utils/timeout.py", line 38, in handle_timeout
    raise AirflowTaskTimeout(self.error_message)
airflow.exceptions.AirflowTaskTimeout: Timeout}}


> python3 error import Dag
> ------------------------
>
>                 Key: AIRFLOW-1539
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1539
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: DAG
>    Affects Versions: Airflow 1.8
>         Environment: docker debian linux 9, conda python3
>            Reporter: Lulu Cheng
>            Priority: Blocker
>
> Seeing the following error when using python3 (fine with python2)
> {code:python}
> Traceback (most recent call last):
>   File "/opt/conda/lib/python3.6/site-packages/airflow/models.py", line 263, in process_file
>     m = imp.load_source(mod_name, filepath)
>   File "/opt/conda/lib/python3.6/imp.py", line 172, in load_source
>     module = _load(spec)
>   File "<frozen importlib._bootstrap>", line 675, in _load
>   File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
>   File "<frozen importlib._bootstrap_external>", line 678, in exec_module
>   File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
>   File "/root/airflow/dags/LoadS3ToHiveTest.py", line 2, in <module>
>     from LoadS3ToHive import default_args, download_s3_to_hdfs_templated_command, generate_download_s3_to_hdfs_task,
\
>   File "/root/airflow/dags/LoadS3ToHive.py", line 74, in <module>
>     globals()[task_name] = DAG(task_name, default_args=default_args, schedule_interval=None)
>   File "/opt/conda/lib/python3.6/site-packages/airflow/models.py", line 2664, in __init__
>     self.fileloc = inspect.getsourcefile(inspect.stack()[1][0])
>   File "/opt/conda/lib/python3.6/inspect.py", line 1465, in stack
>     return getouterframes(sys._getframe(1), context)
>   File "/opt/conda/lib/python3.6/inspect.py", line 1442, in getouterframes
>     frameinfo = (frame,) + getframeinfo(frame, context)
>   File "/opt/conda/lib/python3.6/inspect.py", line 1411, in getframeinfo
>     filename = getsourcefile(frame) or getfile(frame)
>   File "/opt/conda/lib/python3.6/inspect.py", line 666, in getsourcefile
>     if getattr(getmodule(object, filename), '__loader__', None) is not None:
>   File "/opt/conda/lib/python3.6/inspect.py", line 703, in getmodule
>     if ismodule(module) and hasattr(module, '__file__'):
>   File "/opt/conda/lib/python3.6/site-packages/airflow/utils/timeout.py", line 38, in
handle_timeout
>     raise AirflowTaskTimeout(self.error_message)
> airflow.exceptions.AirflowTaskTimeout: Timeout
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message