airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel (Daniel Lamblin) [BDP - Seoul]" <lamb...@coupang.com>
Subject Re: Google Sensor, last updated
Date Mon, 06 Nov 2017 02:10:46 GMT
I think you’d have a little more luck asking on something like stackoverflow, as it seems
most devs are preoccupied by the outstanding issues and preparing a release candidate.
I’m not sure that the default ts_function is your issue, but I think that if you’re trying
to use a custom one, you’ll need to implement it as a function:
https://github.com/apache/incubator-airflow/blob/3fde1043fe56fb0772d0c0f2a1b32105b18f2583/airflow/contrib/sensors/gcs_sensor.py#L102

So your
    ts_func = '{{ execution_date}}'
could be written as
    ts_func=lambda: context['execution_date']

-Daniel
On 10/31/17, 1:01 AM, "Sam Elamin" <hussam.elamin@gmail.com> wrote:

    Hi Everyone
    
    does anyone have any problems creating a
    GoogleCloudStorageObjectUpdatedSensor
    <https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/sensors/gcs_sensor.py>
    ?	
    
    
    im on airflow 1.8 and i am getting an error of
    
    *File
    "/usr/local/lib/python2.7/dist-packages/airflow/contrib/sensors/gcs_sensor.py",
    line 73, in ts_function*
    
    *[2017-10-30 15:27:10,609] {base_task_runner.py:95} INFO - Subtask: return
    context['execution_date'] + context['dag'].schedule_interval*
    
    *[2017-10-30 15:27:10,609] {base_task_runner.py:95} INFO - Subtask:
    TypeError: unsupported operand type(s) for +: 'datetime.datetime' and 'str'*
    
    *[2017-10-30 15:27:15,906] {jobs.py:2097} INFO - Task exited with return
    code 1*
    
    
    i am not overwriting the ts_function so it seems the default ts_function
    isn't working
    
    When I try to overwrite the dag with
    
    ts_fun = '{{ ds }}' or ts_func = '{{ execution_date}}'
    
    I get
    
    *''str' object is not callable'*
    
    Which I assume is because its reading the date as a string instead of a
    datetime object
    
    Any help would be greatly apprecated?
    
    
    Regards
    Sam
    

Mime
View raw message