flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal
Date Tue, 12 Jul 2016 19:55:15 GMT
I'll make the changes to the list of things that we consider public "API"
and also add these to the FLIP Template. Then we should be good to go. I'll
convert the design docs I started to FLIPs in the next few days and will
also ask other people that have authored design docs to move them to FLIPs.

On Fri, 8 Jul 2016 at 17:44 Aljoscha Krettek <aljoscha@apache.org> wrote:

> I was planning to ask people to do that once the process is in place, yes.
>
> Wouldn't it be "forward porting", though? :-)
>
> On Fri, 8 Jul 2016 at 16:38 Robert Metzger <rmetzger@apache.org> wrote:
>
>> What about "backporting" some of the currently ongoing FLIPs to the new
>> process?
>> In particular those which are not yet fully implemented (Security, Mesos,
>> Key Groups, ...) ?
>>
>> On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <aljoscha@apache.org>
>> wrote:
>>
>> > Of course, this would make the list look like this:
>> >
>> > All of the following are public interfaces that people build around:
>> >  - DataStream and DataSet API, including classes related to that, such
>> as
>> > StreamExecutionEnvironment
>> >  - Classes marked with the @Public annotation
>> >  - On-disk binary formats, such as checkpoints/savepoints
>> >  - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
>> >  - Configuration
>> >  - Monitoring
>> >
>> > On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uce@apache.org> wrote:
>> >
>> > > What about the scripts?
>> > >
>> > > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <aljoscha@apache.org
>> >
>> > > wrote:
>> > > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based
on
>> > the
>> > > > template you copied over.
>> > > >
>> > > > I propose to change the list of public interfaces to this:
>> > > >
>> > > > All of the following are public interfaces that people build around:
>> > > >  - DataStream and DataSet API, including classes related to that,
>> such
>> > as
>> > > > StreamExecutionEnvironment
>> > > >  - Classes marked with the @Public annotation
>> > > >  - On-disk binary formats, such as checkpoints/savepoints
>> > > >  - Configuration
>> > > >  - Monitoring
>> > > >
>> > > > Cheers,
>> > > > Aljoscha
>> > > >
>> > > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mjsax@apache.org>
>> wrote:
>> > > >
>> > > >> Done.
>> > > >>
>> > > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are
some
>> things
>> > > >> we need to rework. Please have a look and propose changes.
>> > > >>
>> > > >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>> > > >>
>> > > >> Same for FLIP Template
>> > > >>
>> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
>> > > >>
>> > > >>
>> > > >> -Matthias
>> > > >>
>> > > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
>> > > >> > Hi,
>> > > >> > I got this reply from one of the Kafka committers:
>> > > >> >
>> > > >> > Thanks for sharing your intention to use a process similar
to our
>> > KIP
>> > > >> >
>> > > >> > process. You are more than welcome to copy the structures
and
>> docs
>> > > that
>> > > >> we
>> > > >> >
>> > > >> > have for the KIP process. :)
>> > > >> >
>> > > >> > Ismael
>> > > >> >
>> > > >> >
>> > > >> > So it seems we're good to go. @Matthias, since you are a
Kafka
>> > > >> contributor,
>> > > >> > could you maybe copy the relevant docs from the Kafka wiki
to our
>> > > wiki?
>> > > >> The
>> > > >> > source for this page:
>> > > >> >
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> > > >> > and
>> > > >> > the KIP template that is referenced there. If you want, I
can
>> take
>> > it
>> > > >> from
>> > > >> > there and adapt it to Flink and then let everyone discuss
it
>> again.
>> > > >> >
>> > > >> >
>> > > >> > Cheers,
>> > > >> >
>> > > >> > Aljoscha
>> > > >> >
>> > > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <
>> aljoscha@apache.org>
>> > > >> wrote:
>> > > >> >
>> > > >> >> I'll reach out to the Kafka community and ask if it's
ok for us
>> to
>> > > >> "steal"
>> > > >> >> their process for this.
>> > > >> >>
>> > > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <
>> aljoscha@apache.org>
>> > > >> wrote:
>> > > >> >>
>> > > >> >>> @Matthias: Yes, this is the reason why I like the
KIP process
>> and
>> > > why I
>> > > >> >>> said "The problem with these is that a) the comments
on the
>> Google
>> > > >> Docs are
>> > > >> >>> not reflected in Jira and the mailing list. There
has been some
>> > very
>> > > >> active
>> > > >> >>> discussion on some of the docs that most people would
never
>> > > notice.".
>> > > >> >>>
>> > > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <
>> rmetzger@apache.org>
>> > > >> wrote:
>> > > >> >>>
>> > > >> >>>> I also like the proposal. I think its an issue
that Google
>> Docs
>> > > >> comments
>> > > >> >>>> are not reflected within ASF infra. Therefore,
I'm +1 on
>> > discussing
>> > > >> the
>> > > >> >>>> proposals on the mailing list.
>> > > >> >>>>
>> > > >> >>>> I agree that we need to clean up our wiki.
>> > > >> >>>>
>> > > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J.
Sax <
>> > mjsax@apache.org
>> > > >
>> > > >> >>>> wrote:
>> > > >> >>>>
>> > > >> >>>>> Just to point out one thing about Kafka KIPs
and using the
>> > project
>> > > >> >>>> wiki:
>> > > >> >>>>>
>> > > >> >>>>> The wiki contains the current state of the
proposal, while
>> the
>> > > >> >>>>> discussion is covered over the dev-mailing
list. IMHO, this
>> > makes
>> > > a
>> > > >> lot
>> > > >> >>>>> of sense, as people tend to follow the mailing
list but not
>> wiki
>> > > >> >>>>> changes. Furthermore, the mailing list tracks
the whole
>> > discussion
>> > > >> >>>>> history, while the proposal is kept in a
clean state and thus
>> > > easy to
>> > > >> >>>> read.
>> > > >> >>>>>
>> > > >> >>>>> -Matthias
>> > > >> >>>>>
>> > > >> >>>>>
>> > > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek
wrote:
>> > > >> >>>>>> Jip, that's why I referenced the Kafka
process which is
>> also in
>> > > >> their
>> > > >> >>>>> wiki.
>> > > >> >>>>>>
>> > > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen
<sewen@apache.org>
>> > > wrote:
>> > > >> >>>>>>
>> > > >> >>>>>>> Yes, big +1
>> > > >> >>>>>>>
>> > > >> >>>>>>> I had actually talked about the same
thing with some
>> people as
>> > > >> well.
>> > > >> >>>>>>>
>> > > >> >>>>>>> I am currently sketching a few FLIPs
for things, like
>> > > improvements
>> > > >> >>>> to
>> > > >> >>>>> the
>> > > >> >>>>>>> Yarn/Mesos/Kubernetes integration
>> > > >> >>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>> One thing we should do here is to
actually structure the
>> wiki
>> > a
>> > > bit
>> > > >> >>>> to
>> > > >> >>>>> make
>> > > >> >>>>>>> it easier to find information and
proposals.
>> > > >> >>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk
Celebi <
>> uce@apache.org>
>> > > >> wrote:
>> > > >> >>>>>>>
>> > > >> >>>>>>>> Hey Aljoscha,
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> thanks for this proposal. I've
somehow missed it last
>> week. I
>> > > like
>> > > >> >>>> the
>> > > >> >>>>>>>> idea very much and agree with
your assessment about the
>> > > problems
>> > > >> >>>> with
>> > > >> >>>>>>>> the Google Doc approach.
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> Regarding the process: I'm also
in favour of adopting it
>> from
>> > > >> >>>> Kafka. I
>> > > >> >>>>>>>> would not expect any problems
with this, but we can post a
>> > > quick
>> > > >> >>>> note
>> > > >> >>>>>>>> to their ML.
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> @Matthias: The name works for
me. ;-)
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> – Ufuk
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19
PM, Matthias J. Sax <
>> > > >> >>>> mjsax@apache.org>
>> > > >> >>>>>>>> wrote:
>> > > >> >>>>>>>>> FLIP ?? Really? :D
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>> http://www.maya.tv/en/character/flip
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>> -Matthias
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha
Krettek wrote:
>> > > >> >>>>>>>>>> I'm proposing to add
a formal process for how we deal
>> with
>> > > >> >>>> (major)
>> > > >> >>>>>>>>>> improvements to Flink
and design docs. This has been
>> > > mentioned
>> > > >> >>>>> several
>> > > >> >>>>>>>>>> times recently but we
never took any decisive action to
>> > > actually
>> > > >> >>>>>>>> implement
>> > > >> >>>>>>>>>> such a process so here
we go.
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>> Right now, we have Jira
issues and we sometimes we have
>> > > design
>> > > >> >>>> docs
>> > > >> >>>>>>>> that we
>> > > >> >>>>>>>>>> keep in Google Docs.
Jamie recently added links to those
>> > > that he
>> > > >> >>>>> could
>> > > >> >>>>>>>> find
>> > > >> >>>>>>>>>> on the mailing list to
the Flink wiki:
>> > > >> >>>>>>>>>>
>> > > >> >>>>
>> > > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>> > > >> >>>>>>>> The
>> > > >> >>>>>>>>>> problem with these is
that a) the comments on the Google
>> > Docs
>> > > >> >>>> are not
>> > > >> >>>>>>>>>> reflected in Jira and
the mailing list. There has been
>> some
>> > > very
>> > > >> >>>>>>> active
>> > > >> >>>>>>>>>> discussion on some of
the docs that most people would
>> never
>> > > >> >>>> notice.
>> > > >> >>>>>>> The
>> > > >> >>>>>>>>>> community therefore might
seem less active than it
>> actually
>> > > is.
>> > > >> >>>> b)
>> > > >> >>>>> the
>> > > >> >>>>>>>>>> documents are not very
discoverable, if we had a clearly
>> > > defined
>> > > >> >>>>> place
>> > > >> >>>>>>>>>> where we put them and
also prominently link to this on
>> the
>> > > Flink
>> > > >> >>>>>>>> homepage
>> > > >> >>>>>>>>>> this would greatly help
people that try to find out
>> about
>> > > >> current
>> > > >> >>>>>>>>>> developments.
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>> Kafka has a process like
this:
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>
>> > > >> >>>>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> > > >> >>>>>>>> .
>> > > >> >>>>>>>>>> They call it KIP, for
Kafka Improvement Proposal. We
>> could
>> > > >> either
>> > > >> >>>>>>> adapt
>> > > >> >>>>>>>>>> this for Flink or come
up with our own process. Doing
>> the
>> > > former
>> > > >> >>>>> would
>> > > >> >>>>>>>> save
>> > > >> >>>>>>>>>> us a lot of time and
I don't think the Kafka community
>> > would
>> > > >> >>>> mind us
>> > > >> >>>>>>>>>> copying their process.
The subject also hints at this,
>> our
>> > > >> >>>> process
>> > > >> >>>>>>>> could be
>> > > >> >>>>>>>>>> called FLIP, for Flink
Improvement Proposal.
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>> What do you think? Feedback
is highly welcome. :-)
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>> Cheers,
>> > > >> >>>>>>>>>> Aljoscha
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>
>> > > >> >>>>>
>> > > >> >>>>>
>> > > >> >>>>
>> > > >> >>>
>> > > >> >
>> > > >>
>> > > >>
>> > >
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message