airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ford Guo (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (AIRFLOW-775) AutoCommit in jdbc hook seems not to turn off if set to false
Date Tue, 18 Apr 2017 10:33:41 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ford Guo reassigned AIRFLOW-775:
--------------------------------

    Assignee: Ford Guo

> AutoCommit in jdbc hook seems not to turn off if set to false
> -------------------------------------------------------------
>
>                 Key: AIRFLOW-775
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-775
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: db, hooks
>            Reporter: I don't want an account
>            Assignee: Ford Guo
>
> If I use JdbcHook and run with autocommit=false I still get exceptions when the commit
is made because autocommit mode is on by default and apparently was not set to off.
> This can be worked around by setting the connection host with ;autocommit=false
> however it doesn't seem like the intended behavior when passing autocommit=False with
the hook's methods.
> The JdbcHook does not seem to have a constructor that could take the jdbc driver, location,
host, schema, port, username, and password and work without a set connection id, so working
around this in code isn't too straightforward either.
> [2017-01-19 19:03:22,728] {models.py:1286} ERROR - org.netezza.error.NzSQLException:
The connection object is in auto-commit mode
> Traceback (most recent call last):
>   File "/Users/daniellamblin/VEnvs/airflow-dags/lib/python2.7/site-packages/airflow/models.py",
line 1242, in run
>     result = task_copy.execute(context=context)
>   File "/Users/daniellamblin/VEnvs/airflow-dags/lib/python2.7/site-packages/airflow/operators/python_operator.py",
line 66, in execute
>     return_value = self.python_callable(*self.op_args, **self.op_kwargs)
>   File "/Users/daniellamblin/airflow/dags/dpds/dpds_go_pda_dwd_sku_and_dwd_hist_up_sku_grade.py",
line 356, in stage_to_update_tables
>     hook.run(sql=sql, autocommit=False)
>   File "/Users/daniellamblin/VEnvs/airflow-dags/lib/python2.7/site-packages/airflow/hooks/dbapi_hook.py",
line 134, in run
>     conn.commit()
>   File "/Users/daniellamblin/VEnvs/airflow-dags/lib/python2.7/site-packages/jaydebeapi/__init__.py",
line 391, in commit
>     _handle_sql_exception()
>   File "/Users/daniellamblin/VEnvs/airflow-dags/lib/python2.7/site-packages/jaydebeapi/__init__.py",
line 148, in _handle_sql_exception_jpype
>     reraise(exc_type, exc_info[1], exc_info[2])
>   File "/Users/daniellamblin/VEnvs/airflow-dags/lib/python2.7/site-packages/jaydebeapi/__init__.py",
line 389, in commit
>     self.jconn.commit()
> DatabaseError: org.netezza.error.NzSQLException: The connection object is in auto-commit
mode
> [2017-01-19 19:03:22,730] {models.py:1306} INFO - Marking task as FAILED.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message