flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal
Date Fri, 08 Jul 2016 14:37:59 GMT
What about "backporting" some of the currently ongoing FLIPs to the new
process?
In particular those which are not yet fully implemented (Security, Mesos,
Key Groups, ...) ?

On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <aljoscha@apache.org>
wrote:

> Of course, this would make the list look like this:
>
> All of the following are public interfaces that people build around:
>  - DataStream and DataSet API, including classes related to that, such as
> StreamExecutionEnvironment
>  - Classes marked with the @Public annotation
>  - On-disk binary formats, such as checkpoints/savepoints
>  - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
>  - Configuration
>  - Monitoring
>
> On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uce@apache.org> wrote:
>
> > What about the scripts?
> >
> > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <aljoscha@apache.org>
> > wrote:
> > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based on
> the
> > > template you copied over.
> > >
> > > I propose to change the list of public interfaces to this:
> > >
> > > All of the following are public interfaces that people build around:
> > >  - DataStream and DataSet API, including classes related to that, such
> as
> > > StreamExecutionEnvironment
> > >  - Classes marked with the @Public annotation
> > >  - On-disk binary formats, such as checkpoints/savepoints
> > >  - Configuration
> > >  - Monitoring
> > >
> > > Cheers,
> > > Aljoscha
> > >
> > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mjsax@apache.org> wrote:
> > >
> > >> Done.
> > >>
> > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some things
> > >> we need to rework. Please have a look and propose changes.
> > >>
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
> > >>
> > >> Same for FLIP Template
> > >>
> > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
> > >>
> > >>
> > >> -Matthias
> > >>
> > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
> > >> > Hi,
> > >> > I got this reply from one of the Kafka committers:
> > >> >
> > >> > Thanks for sharing your intention to use a process similar to our
> KIP
> > >> >
> > >> > process. You are more than welcome to copy the structures and docs
> > that
> > >> we
> > >> >
> > >> > have for the KIP process. :)
> > >> >
> > >> > Ismael
> > >> >
> > >> >
> > >> > So it seems we're good to go. @Matthias, since you are a Kafka
> > >> contributor,
> > >> > could you maybe copy the relevant docs from the Kafka wiki to our
> > wiki?
> > >> The
> > >> > source for this page:
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > >> > and
> > >> > the KIP template that is referenced there. If you want, I can take
> it
> > >> from
> > >> > there and adapt it to Flink and then let everyone discuss it again.
> > >> >
> > >> >
> > >> > Cheers,
> > >> >
> > >> > Aljoscha
> > >> >
> > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <aljoscha@apache.org>
> > >> wrote:
> > >> >
> > >> >> I'll reach out to the Kafka community and ask if it's ok for us
to
> > >> "steal"
> > >> >> their process for this.
> > >> >>
> > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <aljoscha@apache.org>
> > >> wrote:
> > >> >>
> > >> >>> @Matthias: Yes, this is the reason why I like the KIP process
and
> > why I
> > >> >>> said "The problem with these is that a) the comments on the
Google
> > >> Docs are
> > >> >>> not reflected in Jira and the mailing list. There has been
some
> very
> > >> active
> > >> >>> discussion on some of the docs that most people would never
> > notice.".
> > >> >>>
> > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rmetzger@apache.org>
> > >> wrote:
> > >> >>>
> > >> >>>> I also like the proposal. I think its an issue that Google
Docs
> > >> comments
> > >> >>>> are not reflected within ASF infra. Therefore, I'm +1
on
> discussing
> > >> the
> > >> >>>> proposals on the mailing list.
> > >> >>>>
> > >> >>>> I agree that we need to clean up our wiki.
> > >> >>>>
> > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <
> mjsax@apache.org
> > >
> > >> >>>> wrote:
> > >> >>>>
> > >> >>>>> Just to point out one thing about Kafka KIPs and using
the
> project
> > >> >>>> wiki:
> > >> >>>>>
> > >> >>>>> The wiki contains the current state of the proposal,
while the
> > >> >>>>> discussion is covered over the dev-mailing list. IMHO,
this
> makes
> > a
> > >> lot
> > >> >>>>> of sense, as people tend to follow the mailing list
but not wiki
> > >> >>>>> changes. Furthermore, the mailing list tracks the
whole
> discussion
> > >> >>>>> history, while the proposal is kept in a clean state
and thus
> > easy to
> > >> >>>> read.
> > >> >>>>>
> > >> >>>>> -Matthias
> > >> >>>>>
> > >> >>>>>
> > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> > >> >>>>>> Jip, that's why I referenced the Kafka process
which is also in
> > >> their
> > >> >>>>> wiki.
> > >> >>>>>>
> > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <sewen@apache.org>
> > wrote:
> > >> >>>>>>
> > >> >>>>>>> Yes, big +1
> > >> >>>>>>>
> > >> >>>>>>> I had actually talked about the same thing
with some people as
> > >> well.
> > >> >>>>>>>
> > >> >>>>>>> I am currently sketching a few FLIPs for things,
like
> > improvements
> > >> >>>> to
> > >> >>>>> the
> > >> >>>>>>> Yarn/Mesos/Kubernetes integration
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> One thing we should do here is to actually
structure the wiki
> a
> > bit
> > >> >>>> to
> > >> >>>>> make
> > >> >>>>>>> it easier to find information and proposals.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi
<uce@apache.org>
> > >> wrote:
> > >> >>>>>>>
> > >> >>>>>>>> Hey Aljoscha,
> > >> >>>>>>>>
> > >> >>>>>>>> thanks for this proposal. I've somehow
missed it last week. I
> > like
> > >> >>>> the
> > >> >>>>>>>> idea very much and agree with your assessment
about the
> > problems
> > >> >>>> with
> > >> >>>>>>>> the Google Doc approach.
> > >> >>>>>>>>
> > >> >>>>>>>> Regarding the process: I'm also in favour
of adopting it from
> > >> >>>> Kafka. I
> > >> >>>>>>>> would not expect any problems with this,
but we can post a
> > quick
> > >> >>>> note
> > >> >>>>>>>> to their ML.
> > >> >>>>>>>>
> > >> >>>>>>>> @Matthias: The name works for me. ;-)
> > >> >>>>>>>>
> > >> >>>>>>>> – Ufuk
> > >> >>>>>>>>
> > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias
J. Sax <
> > >> >>>> mjsax@apache.org>
> > >> >>>>>>>> wrote:
> > >> >>>>>>>>> FLIP ?? Really? :D
> > >> >>>>>>>>>
> > >> >>>>>>>>> http://www.maya.tv/en/character/flip
> > >> >>>>>>>>>
> > >> >>>>>>>>> -Matthias
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek
wrote:
> > >> >>>>>>>>>> I'm proposing to add a formal
process for how we deal with
> > >> >>>> (major)
> > >> >>>>>>>>>> improvements to Flink and design
docs. This has been
> > mentioned
> > >> >>>>> several
> > >> >>>>>>>>>> times recently but we never took
any decisive action to
> > actually
> > >> >>>>>>>> implement
> > >> >>>>>>>>>> such a process so here we go.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Right now, we have Jira issues
and we sometimes we have
> > design
> > >> >>>> docs
> > >> >>>>>>>> that we
> > >> >>>>>>>>>> keep in Google Docs. Jamie recently
added links to those
> > that he
> > >> >>>>> could
> > >> >>>>>>>> find
> > >> >>>>>>>>>> on the mailing list to the Flink
wiki:
> > >> >>>>>>>>>>
> > >> >>>>
> > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> > >> >>>>>>>> The
> > >> >>>>>>>>>> problem with these is that a)
the comments on the Google
> Docs
> > >> >>>> are not
> > >> >>>>>>>>>> reflected in Jira and the mailing
list. There has been some
> > very
> > >> >>>>>>> active
> > >> >>>>>>>>>> discussion on some of the docs
that most people would never
> > >> >>>> notice.
> > >> >>>>>>> The
> > >> >>>>>>>>>> community therefore might seem
less active than it actually
> > is.
> > >> >>>> b)
> > >> >>>>> the
> > >> >>>>>>>>>> documents are not very discoverable,
if we had a clearly
> > defined
> > >> >>>>> place
> > >> >>>>>>>>>> where we put them and also prominently
link to this on the
> > Flink
> > >> >>>>>>>> homepage
> > >> >>>>>>>>>> this would greatly help people
that try to find out about
> > >> current
> > >> >>>>>>>>>> developments.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Kafka has a process like this:
> > >> >>>>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>
> > >> >>>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > >> >>>>>>>> .
> > >> >>>>>>>>>> They call it KIP, for Kafka Improvement
Proposal. We could
> > >> either
> > >> >>>>>>> adapt
> > >> >>>>>>>>>> this for Flink or come up with
our own process. Doing the
> > former
> > >> >>>>> would
> > >> >>>>>>>> save
> > >> >>>>>>>>>> us a lot of time and I don't think
the Kafka community
> would
> > >> >>>> mind us
> > >> >>>>>>>>>> copying their process. The subject
also hints at this, our
> > >> >>>> process
> > >> >>>>>>>> could be
> > >> >>>>>>>>>> called FLIP, for Flink Improvement
Proposal.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> What do you think? Feedback is
highly welcome. :-)
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Cheers,
> > >> >>>>>>>>>> Aljoscha
> > >> >>>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>
> > >> >>>
> > >> >
> > >>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message