flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal
Date Fri, 08 Jul 2016 15:44:56 GMT
I was planning to ask people to do that once the process is in place, yes.

Wouldn't it be "forward porting", though? :-)

On Fri, 8 Jul 2016 at 16:38 Robert Metzger <rmetzger@apache.org> wrote:

> What about "backporting" some of the currently ongoing FLIPs to the new
> process?
> In particular those which are not yet fully implemented (Security, Mesos,
> Key Groups, ...) ?
>
> On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <aljoscha@apache.org>
> wrote:
>
> > Of course, this would make the list look like this:
> >
> > All of the following are public interfaces that people build around:
> >  - DataStream and DataSet API, including classes related to that, such as
> > StreamExecutionEnvironment
> >  - Classes marked with the @Public annotation
> >  - On-disk binary formats, such as checkpoints/savepoints
> >  - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
> >  - Configuration
> >  - Monitoring
> >
> > On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uce@apache.org> wrote:
> >
> > > What about the scripts?
> > >
> > > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <aljoscha@apache.org>
> > > wrote:
> > > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based on
> > the
> > > > template you copied over.
> > > >
> > > > I propose to change the list of public interfaces to this:
> > > >
> > > > All of the following are public interfaces that people build around:
> > > >  - DataStream and DataSet API, including classes related to that,
> such
> > as
> > > > StreamExecutionEnvironment
> > > >  - Classes marked with the @Public annotation
> > > >  - On-disk binary formats, such as checkpoints/savepoints
> > > >  - Configuration
> > > >  - Monitoring
> > > >
> > > > Cheers,
> > > > Aljoscha
> > > >
> > > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mjsax@apache.org>
> wrote:
> > > >
> > > >> Done.
> > > >>
> > > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some
> things
> > > >> we need to rework. Please have a look and propose changes.
> > > >>
> > > >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
> > > >>
> > > >> Same for FLIP Template
> > > >>
> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
> > > >>
> > > >>
> > > >> -Matthias
> > > >>
> > > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
> > > >> > Hi,
> > > >> > I got this reply from one of the Kafka committers:
> > > >> >
> > > >> > Thanks for sharing your intention to use a process similar to
our
> > KIP
> > > >> >
> > > >> > process. You are more than welcome to copy the structures and
docs
> > > that
> > > >> we
> > > >> >
> > > >> > have for the KIP process. :)
> > > >> >
> > > >> > Ismael
> > > >> >
> > > >> >
> > > >> > So it seems we're good to go. @Matthias, since you are a Kafka
> > > >> contributor,
> > > >> > could you maybe copy the relevant docs from the Kafka wiki to
our
> > > wiki?
> > > >> The
> > > >> > source for this page:
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > > >> > and
> > > >> > the KIP template that is referenced there. If you want, I can
take
> > it
> > > >> from
> > > >> > there and adapt it to Flink and then let everyone discuss it
> again.
> > > >> >
> > > >> >
> > > >> > Cheers,
> > > >> >
> > > >> > Aljoscha
> > > >> >
> > > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <aljoscha@apache.org
> >
> > > >> wrote:
> > > >> >
> > > >> >> I'll reach out to the Kafka community and ask if it's ok
for us
> to
> > > >> "steal"
> > > >> >> their process for this.
> > > >> >>
> > > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <
> aljoscha@apache.org>
> > > >> wrote:
> > > >> >>
> > > >> >>> @Matthias: Yes, this is the reason why I like the KIP
process
> and
> > > why I
> > > >> >>> said "The problem with these is that a) the comments
on the
> Google
> > > >> Docs are
> > > >> >>> not reflected in Jira and the mailing list. There has
been some
> > very
> > > >> active
> > > >> >>> discussion on some of the docs that most people would
never
> > > notice.".
> > > >> >>>
> > > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rmetzger@apache.org
> >
> > > >> wrote:
> > > >> >>>
> > > >> >>>> I also like the proposal. I think its an issue that
Google Docs
> > > >> comments
> > > >> >>>> are not reflected within ASF infra. Therefore, I'm
+1 on
> > discussing
> > > >> the
> > > >> >>>> proposals on the mailing list.
> > > >> >>>>
> > > >> >>>> I agree that we need to clean up our wiki.
> > > >> >>>>
> > > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax
<
> > mjsax@apache.org
> > > >
> > > >> >>>> wrote:
> > > >> >>>>
> > > >> >>>>> Just to point out one thing about Kafka KIPs
and using the
> > project
> > > >> >>>> wiki:
> > > >> >>>>>
> > > >> >>>>> The wiki contains the current state of the proposal,
while the
> > > >> >>>>> discussion is covered over the dev-mailing list.
IMHO, this
> > makes
> > > a
> > > >> lot
> > > >> >>>>> of sense, as people tend to follow the mailing
list but not
> wiki
> > > >> >>>>> changes. Furthermore, the mailing list tracks
the whole
> > discussion
> > > >> >>>>> history, while the proposal is kept in a clean
state and thus
> > > easy to
> > > >> >>>> read.
> > > >> >>>>>
> > > >> >>>>> -Matthias
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> > > >> >>>>>> Jip, that's why I referenced the Kafka process
which is also
> in
> > > >> their
> > > >> >>>>> wiki.
> > > >> >>>>>>
> > > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen
<sewen@apache.org>
> > > wrote:
> > > >> >>>>>>
> > > >> >>>>>>> Yes, big +1
> > > >> >>>>>>>
> > > >> >>>>>>> I had actually talked about the same
thing with some people
> as
> > > >> well.
> > > >> >>>>>>>
> > > >> >>>>>>> I am currently sketching a few FLIPs
for things, like
> > > improvements
> > > >> >>>> to
> > > >> >>>>> the
> > > >> >>>>>>> Yarn/Mesos/Kubernetes integration
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> One thing we should do here is to actually
structure the
> wiki
> > a
> > > bit
> > > >> >>>> to
> > > >> >>>>> make
> > > >> >>>>>>> it easier to find information and proposals.
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk
Celebi <uce@apache.org
> >
> > > >> wrote:
> > > >> >>>>>>>
> > > >> >>>>>>>> Hey Aljoscha,
> > > >> >>>>>>>>
> > > >> >>>>>>>> thanks for this proposal. I've somehow
missed it last
> week. I
> > > like
> > > >> >>>> the
> > > >> >>>>>>>> idea very much and agree with your
assessment about the
> > > problems
> > > >> >>>> with
> > > >> >>>>>>>> the Google Doc approach.
> > > >> >>>>>>>>
> > > >> >>>>>>>> Regarding the process: I'm also in
favour of adopting it
> from
> > > >> >>>> Kafka. I
> > > >> >>>>>>>> would not expect any problems with
this, but we can post a
> > > quick
> > > >> >>>> note
> > > >> >>>>>>>> to their ML.
> > > >> >>>>>>>>
> > > >> >>>>>>>> @Matthias: The name works for me.
;-)
> > > >> >>>>>>>>
> > > >> >>>>>>>> – Ufuk
> > > >> >>>>>>>>
> > > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM,
Matthias J. Sax <
> > > >> >>>> mjsax@apache.org>
> > > >> >>>>>>>> wrote:
> > > >> >>>>>>>>> FLIP ?? Really? :D
> > > >> >>>>>>>>>
> > > >> >>>>>>>>> http://www.maya.tv/en/character/flip
> > > >> >>>>>>>>>
> > > >> >>>>>>>>> -Matthias
> > > >> >>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha
Krettek wrote:
> > > >> >>>>>>>>>> I'm proposing to add a formal
process for how we deal
> with
> > > >> >>>> (major)
> > > >> >>>>>>>>>> improvements to Flink and
design docs. This has been
> > > mentioned
> > > >> >>>>> several
> > > >> >>>>>>>>>> times recently but we never
took any decisive action to
> > > actually
> > > >> >>>>>>>> implement
> > > >> >>>>>>>>>> such a process so here we
go.
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Right now, we have Jira issues
and we sometimes we have
> > > design
> > > >> >>>> docs
> > > >> >>>>>>>> that we
> > > >> >>>>>>>>>> keep in Google Docs. Jamie
recently added links to those
> > > that he
> > > >> >>>>> could
> > > >> >>>>>>>> find
> > > >> >>>>>>>>>> on the mailing list to the
Flink wiki:
> > > >> >>>>>>>>>>
> > > >> >>>>
> > > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> > > >> >>>>>>>> The
> > > >> >>>>>>>>>> problem with these is that
a) the comments on the Google
> > Docs
> > > >> >>>> are not
> > > >> >>>>>>>>>> reflected in Jira and the
mailing list. There has been
> some
> > > very
> > > >> >>>>>>> active
> > > >> >>>>>>>>>> discussion on some of the
docs that most people would
> never
> > > >> >>>> notice.
> > > >> >>>>>>> The
> > > >> >>>>>>>>>> community therefore might
seem less active than it
> actually
> > > is.
> > > >> >>>> b)
> > > >> >>>>> the
> > > >> >>>>>>>>>> documents are not very discoverable,
if we had a clearly
> > > defined
> > > >> >>>>> place
> > > >> >>>>>>>>>> where we put them and also
prominently link to this on
> the
> > > Flink
> > > >> >>>>>>>> homepage
> > > >> >>>>>>>>>> this would greatly help people
that try to find out about
> > > >> current
> > > >> >>>>>>>>>> developments.
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Kafka has a process like
this:
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>
> > > >> >>>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > > >> >>>>>>>> .
> > > >> >>>>>>>>>> They call it KIP, for Kafka
Improvement Proposal. We
> could
> > > >> either
> > > >> >>>>>>> adapt
> > > >> >>>>>>>>>> this for Flink or come up
with our own process. Doing the
> > > former
> > > >> >>>>> would
> > > >> >>>>>>>> save
> > > >> >>>>>>>>>> us a lot of time and I don't
think the Kafka community
> > would
> > > >> >>>> mind us
> > > >> >>>>>>>>>> copying their process. The
subject also hints at this,
> our
> > > >> >>>> process
> > > >> >>>>>>>> could be
> > > >> >>>>>>>>>> called FLIP, for Flink Improvement
Proposal.
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> What do you think? Feedback
is highly welcome. :-)
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Cheers,
> > > >> >>>>>>>>>> Aljoscha
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >
> > > >>
> > > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message