airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Dolan <peterdo...@google.com.INVALID>
Subject Re: Cloud ML Operators
Date Thu, 29 Jun 2017 16:34:07 GMT
Yes, thanks Chris! I really appreciate your responsiveness especially
during a release. We have one more PR incoming, which we held off because
there was so much code in common. It should be smaller than the previous
ones.

On Thu, Jun 29, 2017 at 8:58 AM Chris Riccomini <criccomini@apache.org>
wrote:

> I think we're all good here. :)
>
> On Mon, Jun 26, 2017 at 2:48 PM, Chris Riccomini <criccomini@apache.org>
> wrote:
>
> > Hey Peter,
> >
> > Thanks, will have a look. I'm a bit tied up with the 1.8.2 release at the
> > moment, so it might be a little longer than usual, but I'll get there. I
> > did do a bunch of Google PRs on Friday, but I must have missed this one.
> > Stay tuned. If you don't get enough response by Wednesday, please
> re-ping.
> >
> > Cheers,
> > Chris
> >
> > On Mon, Jun 26, 2017 at 1:46 PM, Peter Dolan <
> > peterdolan@google.com.invalid> wrote:
> >
> >> Hey Chris,
> >>
> >> We sent our first pull request out for review last week, let me know if
> >> you or someone else has time to take a look at it -
> >> https://github.com/apache/incubator-airflow/pull/2390. Definitely
> >> appreciate quick feedback as this is our first PR and I'm sure we're
> making
> >> some novice errors.
> >>
> >> Thanks,
> >> Peter
> >>
> >> On Mon, Jun 5, 2017 at 1:32 PM Chris Riccomini <criccomini@apache.org>
> >> wrote:
> >>
> >>> Awesome, feel free to ping for code reviews if you don't get them in a
> >>> timely manner.
> >>>
> >>> On Fri, Jun 2, 2017 at 2:46 PM, Peter Dolan
> >>> <peterdolan@google.com.invalid>
> >>> wrote:
> >>>
> >>> > Just created https://issues.apache.org/jira/browse/AIRFLOW-1273,
> >>> > https://issues.apache.org/jira/browse/AIRFLOW-1272, and
> >>> > https://issues.apache.org/jira/browse/AIRFLOW-1271. We'll follow up
> >>> with
> >>> > pull requests in the coming weeks.
> >>> >
> >>> > On Thu, Jun 1, 2017 at 10:44 AM Peter Dolan <peterdolan@google.com>
> >>> wrote:
> >>> >
> >>> >> Hi Bolke,
> >>> >>
> >>> >> Great! And yes, we will be able to maintain these operators after
> the
> >>> >> contributions. We further would always insist on strong test
> coverage
> >>> as
> >>> >> well.
> >>> >>
> >>> >> I'll open some JIRA tickets to track their implementation.
> >>> >>
> >>> >> Thanks,
> >>> >> Peter
> >>> >>
> >>> >> On Thu, Jun 1, 2017 at 6:05 AM Bolke de Bruin <bdbruin@gmail.com>
> >>> wrote:
> >>> >>
> >>> >>> Hi Peter,
> >>> >>>
> >>> >>> That sounds great! I think the main criteria for this is will
you
> >>> >>> maintain the code afterwards? The contrib section is slowly
but
> >>> steadily
> >>> >>> growing and with operators/hooks we are particularly dependent
on
> the
> >>> >>> community as not all (or even none in some case) of the committers
> >>> use
> >>> >>> these themselves.
> >>> >>>
> >>> >>> In any case test coverage is required, but that is a given
I think.
> >>> >>>
> >>> >>> Kind regards,
> >>> >>> Bolke
> >>> >>>
> >>> >>> > On 31 May 2017, at 21:10, Peter Dolan
> >>> <peterdolan@google.com.INVALID>
> >>> >>> wrote:
> >>> >>> >
> >>> >>> > Hello developers,
> >>> >>> >
> >>> >>> > I work with Google Cloud ML, and my team and I are interested
in
> >>> >>> contributing a set of Operators to support working with the
Cloud
> ML
> >>> >>> platform. The platform supports using the TensorFlow deep neural
> >>> network
> >>> >>> framework as a managed system.
> >>> >>> >
> >>> >>> > In particular, we would like to contribute
> >>> >>> >  * CloudMLTrainingOperator, which would launch and monitor
a
> Cloud
> >>> ML
> >>> >>> Training Job (https://cloud.google.com/ml-e
> >>> ngine/docs/how-tos/training-
> >>> >>> jobs <
> https://cloud.google.com/ml-engine/docs/how-tos/training-jobs
> >>> >),
> >>> >>> >  * CloudMLBatchPredictionOperator, which would launch
and monitor
> >>> a
> >>> >>> Cloud ML Batch Prediction Job (https://cloud.google.com/ml-
> >>> >>> engine/docs/how-tos/batch-predict <https://cloud.google.com/ml-
> >>> >>> engine/docs/how-tos/batch-predict>), and
> >>> >>> >  * CloudMLVersionOperator, which can create, update, and
delete
> >>> >>> TensorFlow model versions (https://cloud.google.com/ml-
> >>> >>> engine/docs/how-tos/managing-models-jobs <
> >>> https://cloud.google.com/ml-
> >>> >>> engine/docs/how-tos/managing-models-jobs>)
> >>> >>> >
> >>> >>> > I'm eager to hear if the Airflow project is open to these
> >>> >>> contributions, and if any changes are suggested. We have working
> >>> prototype
> >>> >>> versions of all of them.
> >>> >>> >
> >>> >>> > Thanks in advance,
> >>> >>> > Peter
> >>> >>>
> >>> >>>
> >>>
> >>
> >
>

Mime
View raw message