airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Cloud ML Operators
Date Mon, 26 Jun 2017 21:48:52 GMT
Hey Peter,

Thanks, will have a look. I'm a bit tied up with the 1.8.2 release at the
moment, so it might be a little longer than usual, but I'll get there. I
did do a bunch of Google PRs on Friday, but I must have missed this one.
Stay tuned. If you don't get enough response by Wednesday, please re-ping.

Cheers,
Chris

On Mon, Jun 26, 2017 at 1:46 PM, Peter Dolan <peterdolan@google.com.invalid>
wrote:

> Hey Chris,
>
> We sent our first pull request out for review last week, let me know if
> you or someone else has time to take a look at it -
> https://github.com/apache/incubator-airflow/pull/2390. Definitely
> appreciate quick feedback as this is our first PR and I'm sure we're making
> some novice errors.
>
> Thanks,
> Peter
>
> On Mon, Jun 5, 2017 at 1:32 PM Chris Riccomini <criccomini@apache.org>
> wrote:
>
>> Awesome, feel free to ping for code reviews if you don't get them in a
>> timely manner.
>>
>> On Fri, Jun 2, 2017 at 2:46 PM, Peter Dolan <peterdolan@google.com.invalid
>> >
>> wrote:
>>
>> > Just created https://issues.apache.org/jira/browse/AIRFLOW-1273,
>> > https://issues.apache.org/jira/browse/AIRFLOW-1272, and
>> > https://issues.apache.org/jira/browse/AIRFLOW-1271. We'll follow up
>> with
>> > pull requests in the coming weeks.
>> >
>> > On Thu, Jun 1, 2017 at 10:44 AM Peter Dolan <peterdolan@google.com>
>> wrote:
>> >
>> >> Hi Bolke,
>> >>
>> >> Great! And yes, we will be able to maintain these operators after the
>> >> contributions. We further would always insist on strong test coverage
>> as
>> >> well.
>> >>
>> >> I'll open some JIRA tickets to track their implementation.
>> >>
>> >> Thanks,
>> >> Peter
>> >>
>> >> On Thu, Jun 1, 2017 at 6:05 AM Bolke de Bruin <bdbruin@gmail.com>
>> wrote:
>> >>
>> >>> Hi Peter,
>> >>>
>> >>> That sounds great! I think the main criteria for this is will you
>> >>> maintain the code afterwards? The contrib section is slowly but
>> steadily
>> >>> growing and with operators/hooks we are particularly dependent on the
>> >>> community as not all (or even none in some case) of the committers use
>> >>> these themselves.
>> >>>
>> >>> In any case test coverage is required, but that is a given I think.
>> >>>
>> >>> Kind regards,
>> >>> Bolke
>> >>>
>> >>> > On 31 May 2017, at 21:10, Peter Dolan <peterdolan@google.com.INVALID
>> >
>> >>> wrote:
>> >>> >
>> >>> > Hello developers,
>> >>> >
>> >>> > I work with Google Cloud ML, and my team and I are interested in
>> >>> contributing a set of Operators to support working with the Cloud ML
>> >>> platform. The platform supports using the TensorFlow deep neural
>> network
>> >>> framework as a managed system.
>> >>> >
>> >>> > In particular, we would like to contribute
>> >>> >  * CloudMLTrainingOperator, which would launch and monitor a Cloud
>> ML
>> >>> Training Job (https://cloud.google.com/ml-
>> engine/docs/how-tos/training-
>> >>> jobs <https://cloud.google.com/ml-engine/docs/how-tos/training-jobs
>> >),
>> >>> >  * CloudMLBatchPredictionOperator, which would launch and monitor
a
>> >>> Cloud ML Batch Prediction Job (https://cloud.google.com/ml-
>> >>> engine/docs/how-tos/batch-predict <https://cloud.google.com/ml-
>> >>> engine/docs/how-tos/batch-predict>), and
>> >>> >  * CloudMLVersionOperator, which can create, update, and delete
>> >>> TensorFlow model versions (https://cloud.google.com/ml-
>> >>> engine/docs/how-tos/managing-models-jobs <
>> https://cloud.google.com/ml-
>> >>> engine/docs/how-tos/managing-models-jobs>)
>> >>> >
>> >>> > I'm eager to hear if the Airflow project is open to these
>> >>> contributions, and if any changes are suggested. We have working
>> prototype
>> >>> versions of all of them.
>> >>> >
>> >>> > Thanks in advance,
>> >>> > Peter
>> >>>
>> >>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message