airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Cloud ML Operators
Date Mon, 05 Jun 2017 20:32:03 GMT
Awesome, feel free to ping for code reviews if you don't get them in a
timely manner.

On Fri, Jun 2, 2017 at 2:46 PM, Peter Dolan <peterdolan@google.com.invalid>
wrote:

> Just created https://issues.apache.org/jira/browse/AIRFLOW-1273,
> https://issues.apache.org/jira/browse/AIRFLOW-1272, and
> https://issues.apache.org/jira/browse/AIRFLOW-1271. We'll follow up with
> pull requests in the coming weeks.
>
> On Thu, Jun 1, 2017 at 10:44 AM Peter Dolan <peterdolan@google.com> wrote:
>
>> Hi Bolke,
>>
>> Great! And yes, we will be able to maintain these operators after the
>> contributions. We further would always insist on strong test coverage as
>> well.
>>
>> I'll open some JIRA tickets to track their implementation.
>>
>> Thanks,
>> Peter
>>
>> On Thu, Jun 1, 2017 at 6:05 AM Bolke de Bruin <bdbruin@gmail.com> wrote:
>>
>>> Hi Peter,
>>>
>>> That sounds great! I think the main criteria for this is will you
>>> maintain the code afterwards? The contrib section is slowly but steadily
>>> growing and with operators/hooks we are particularly dependent on the
>>> community as not all (or even none in some case) of the committers use
>>> these themselves.
>>>
>>> In any case test coverage is required, but that is a given I think.
>>>
>>> Kind regards,
>>> Bolke
>>>
>>> > On 31 May 2017, at 21:10, Peter Dolan <peterdolan@google.com.INVALID>
>>> wrote:
>>> >
>>> > Hello developers,
>>> >
>>> > I work with Google Cloud ML, and my team and I are interested in
>>> contributing a set of Operators to support working with the Cloud ML
>>> platform. The platform supports using the TensorFlow deep neural network
>>> framework as a managed system.
>>> >
>>> > In particular, we would like to contribute
>>> >  * CloudMLTrainingOperator, which would launch and monitor a Cloud ML
>>> Training Job (https://cloud.google.com/ml-engine/docs/how-tos/training-
>>> jobs <https://cloud.google.com/ml-engine/docs/how-tos/training-jobs>),
>>> >  * CloudMLBatchPredictionOperator, which would launch and monitor a
>>> Cloud ML Batch Prediction Job (https://cloud.google.com/ml-
>>> engine/docs/how-tos/batch-predict <https://cloud.google.com/ml-
>>> engine/docs/how-tos/batch-predict>), and
>>> >  * CloudMLVersionOperator, which can create, update, and delete
>>> TensorFlow model versions (https://cloud.google.com/ml-
>>> engine/docs/how-tos/managing-models-jobs <https://cloud.google.com/ml-
>>> engine/docs/how-tos/managing-models-jobs>)
>>> >
>>> > I'm eager to hear if the Airflow project is open to these
>>> contributions, and if any changes are suggested. We have working prototype
>>> versions of all of them.
>>> >
>>> > Thanks in advance,
>>> > Peter
>>>
>>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message