airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bernardo Najlis <>
Subject AIRFLOW-2224: Add support for CSV files in mysql_to_gcs operator
Date Sat, 24 Mar 2018 17:46:22 GMT

I submitted a PR associated with AIRFLOW-2224
<> last week that
updates the mysql_to_gcs contrib operator, adding support for CSV files
(currently only supports JSON).

One of the Travis CI tests failed and I'm having some challenges
understanding what the problem is: went through the test output log a
couple of times and it seems the issue is with a test completely unrelated
to my changes.

Is this something that anybody has seen before? Currently working to create
a test for my added functionality (which was missing from the original PR)
but I'm getting concerned that this unrelated test that failed before may
fail once again...

Here's a link to the failed test job
<> and a
summary of (what I think) the error is:

[2018-03-18 23:56:15,972] {} ERROR -
(_mysql_exceptions.IntegrityError) (1062, "Duplicate entry
'example_bash_operator-2018-03-18 23:56:15.000000' for key 'dag_id'") [SQL:
'INSERT INTO dag_run (dag_id, execution_date, start_date, end_date, state,
run_id, external_trigger, conf) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)']
[parameters: ('example_bash_operator', datetime.datetime(2018, 3, 18, 23,
56, 15, tzinfo=datetime.timezone.utc), datetime.datetime(2018, 3, 18, 23,
56, 15, 971501, tzinfo=datetime.timezone.utc), None, 'running',
'my_run2018-03-18T23:56:15.605960', 1, None)]

Any kind of help is greatly appreciated, really interested in getting this
PR into master, and also adding the same functionality (CSV file support)
to multiple other operators.


Bernardo Najlis

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message