airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-2184) Create a druid_checker operator
Date Fri, 09 Mar 2018 21:23:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-2184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393610#comment-16393610
] 

ASF subversion and git services commented on AIRFLOW-2184:
----------------------------------------------------------

Commit 803767959e27ea2f8424fd5afa40f1bab9f15648 in incubator-airflow's branch refs/heads/master
from [~danielkl3]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=8037679 ]

[AIRFLOW-2184][AIRFLOW-2138] Google Cloud Storage allow wildcards

- closes #2184

- Add support for wildcards to be provided in
source object argument
This allows the user of the Operator to provide a
wildcard in the format accepted by the
documentation. This message is echoed in the
docstring for ease of use, and also because it is
only three sentences and adding a link is not
required.
- Add an argument move_object (bool) to the
operator that, when true runs a mv command as
opposed to a cp command. That is to say, it moves
an object instead of copying the object. This is
especially useful when this operator is used to
move objects in the same bucket, perhaps from
folder to folder.
- Add dotmodus and dannylee12 to companies using
airflow
We use airflow in almost all of our projects.

-Unit tests written for the 3 use cases of the
added operator.

Remove newline

Split too long line over 2 lines.

Closes #3067 from DannyLee12/master


> Create a druid_checker operator
> -------------------------------
>
>                 Key: AIRFLOW-2184
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2184
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Tao Feng
>            Assignee: Tao Feng
>            Priority: Major
>
> Once we agree on the extended interface provided through druid_hook in AIRFLOW-2183,
we would like to create a druid_checker operator to do basic data quality checking on data
in druid.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message