airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Elamin <hussam.ela...@gmail.com>
Subject Re: Google Sensor, last updated
Date Mon, 06 Nov 2017 15:02:12 GMT
Hi Daniel

Thanks for the email. It was a bug in the contrib sensor

I raised a pr

https://github.com/apache/incubator-airflow/pull/2747

Regards
Sam

On Mon, Nov 6, 2017 at 2:10 AM, Daniel (Daniel Lamblin) [BDP - Seoul] <
lamblin@coupang.com> wrote:

> I think you’d have a little more luck asking on something like
> stackoverflow, as it seems most devs are preoccupied by the outstanding
> issues and preparing a release candidate.
> I’m not sure that the default ts_function is your issue, but I think that
> if you’re trying to use a custom one, you’ll need to implement it as a
> function:
> https://github.com/apache/incubator-airflow/blob/
> 3fde1043fe56fb0772d0c0f2a1b32105b18f2583/airflow/contrib/
> sensors/gcs_sensor.py#L102
>
> So your
>     ts_func = '{{ execution_date}}'
> could be written as
>     ts_func=lambda: context['execution_date']
>
> -Daniel
> On 10/31/17, 1:01 AM, "Sam Elamin" <hussam.elamin@gmail.com> wrote:
>
>     Hi Everyone
>
>     does anyone have any problems creating a
>     GoogleCloudStorageObjectUpdatedSensor
>     <https://github.com/apache/incubator-airflow/blob/master/
> airflow/contrib/sensors/gcs_sensor.py>
>     ?
>
>
>     im on airflow 1.8 and i am getting an error of
>
>     *File
>     "/usr/local/lib/python2.7/dist-packages/airflow/contrib/
> sensors/gcs_sensor.py",
>     line 73, in ts_function*
>
>     *[2017-10-30 15:27:10,609] {base_task_runner.py:95} INFO - Subtask:
> return
>     context['execution_date'] + context['dag'].schedule_interval*
>
>     *[2017-10-30 15:27:10,609] {base_task_runner.py:95} INFO - Subtask:
>     TypeError: unsupported operand type(s) for +: 'datetime.datetime' and
> 'str'*
>
>     *[2017-10-30 15:27:15,906] {jobs.py:2097} INFO - Task exited with
> return
>     code 1*
>
>
>     i am not overwriting the ts_function so it seems the default
> ts_function
>     isn't working
>
>     When I try to overwrite the dag with
>
>     ts_fun = '{{ ds }}' or ts_func = '{{ execution_date}}'
>
>     I get
>
>     *''str' object is not callable'*
>
>     Which I assume is because its reading the date as a string instead of a
>     datetime object
>
>     Any help would be greatly apprecated?
>
>
>     Regards
>     Sam
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message