flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal
Date Tue, 19 Jul 2016 09:22:13 GMT
Great initiative! I think mailing list discussions can be very hard to
follow in retrospective. The Kafka community does a great job in
summing up the discussions on individual wiki pages. We should do that
as well!

On Tue, Jul 12, 2016 at 9:55 PM, Aljoscha Krettek <aljoscha@apache.org> wrote:
> I'll make the changes to the list of things that we consider public "API"
> and also add these to the FLIP Template. Then we should be good to go. I'll
> convert the design docs I started to FLIPs in the next few days and will
> also ask other people that have authored design docs to move them to FLIPs.
>
> On Fri, 8 Jul 2016 at 17:44 Aljoscha Krettek <aljoscha@apache.org> wrote:
>
>> I was planning to ask people to do that once the process is in place, yes.
>>
>> Wouldn't it be "forward porting", though? :-)
>>
>> On Fri, 8 Jul 2016 at 16:38 Robert Metzger <rmetzger@apache.org> wrote:
>>
>>> What about "backporting" some of the currently ongoing FLIPs to the new
>>> process?
>>> In particular those which are not yet fully implemented (Security, Mesos,
>>> Key Groups, ...) ?
>>>
>>> On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <aljoscha@apache.org>
>>> wrote:
>>>
>>> > Of course, this would make the list look like this:
>>> >
>>> > All of the following are public interfaces that people build around:
>>> >  - DataStream and DataSet API, including classes related to that, such
>>> as
>>> > StreamExecutionEnvironment
>>> >  - Classes marked with the @Public annotation
>>> >  - On-disk binary formats, such as checkpoints/savepoints
>>> >  - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
>>> >  - Configuration
>>> >  - Monitoring
>>> >
>>> > On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uce@apache.org> wrote:
>>> >
>>> > > What about the scripts?
>>> > >
>>> > > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <aljoscha@apache.org
>>> >
>>> > > wrote:
>>> > > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based
on
>>> > the
>>> > > > template you copied over.
>>> > > >
>>> > > > I propose to change the list of public interfaces to this:
>>> > > >
>>> > > > All of the following are public interfaces that people build around:
>>> > > >  - DataStream and DataSet API, including classes related to that,
>>> such
>>> > as
>>> > > > StreamExecutionEnvironment
>>> > > >  - Classes marked with the @Public annotation
>>> > > >  - On-disk binary formats, such as checkpoints/savepoints
>>> > > >  - Configuration
>>> > > >  - Monitoring
>>> > > >
>>> > > > Cheers,
>>> > > > Aljoscha
>>> > > >
>>> > > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mjsax@apache.org>
>>> wrote:
>>> > > >
>>> > > >> Done.
>>> > > >>
>>> > > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there
are some
>>> things
>>> > > >> we need to rework. Please have a look and propose changes.
>>> > > >>
>>> > > >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>>> > > >>
>>> > > >> Same for FLIP Template
>>> > > >>
>>> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
>>> > > >>
>>> > > >>
>>> > > >> -Matthias
>>> > > >>
>>> > > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
>>> > > >> > Hi,
>>> > > >> > I got this reply from one of the Kafka committers:
>>> > > >> >
>>> > > >> > Thanks for sharing your intention to use a process similar
to our
>>> > KIP
>>> > > >> >
>>> > > >> > process. You are more than welcome to copy the structures
and
>>> docs
>>> > > that
>>> > > >> we
>>> > > >> >
>>> > > >> > have for the KIP process. :)
>>> > > >> >
>>> > > >> > Ismael
>>> > > >> >
>>> > > >> >
>>> > > >> > So it seems we're good to go. @Matthias, since you are
a Kafka
>>> > > >> contributor,
>>> > > >> > could you maybe copy the relevant docs from the Kafka
wiki to our
>>> > > wiki?
>>> > > >> The
>>> > > >> > source for this page:
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>>> > > >> > and
>>> > > >> > the KIP template that is referenced there. If you want,
I can
>>> take
>>> > it
>>> > > >> from
>>> > > >> > there and adapt it to Flink and then let everyone discuss
it
>>> again.
>>> > > >> >
>>> > > >> >
>>> > > >> > Cheers,
>>> > > >> >
>>> > > >> > Aljoscha
>>> > > >> >
>>> > > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <
>>> aljoscha@apache.org>
>>> > > >> wrote:
>>> > > >> >
>>> > > >> >> I'll reach out to the Kafka community and ask if
it's ok for us
>>> to
>>> > > >> "steal"
>>> > > >> >> their process for this.
>>> > > >> >>
>>> > > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <
>>> aljoscha@apache.org>
>>> > > >> wrote:
>>> > > >> >>
>>> > > >> >>> @Matthias: Yes, this is the reason why I like
the KIP process
>>> and
>>> > > why I
>>> > > >> >>> said "The problem with these is that a) the comments
on the
>>> Google
>>> > > >> Docs are
>>> > > >> >>> not reflected in Jira and the mailing list. There
has been some
>>> > very
>>> > > >> active
>>> > > >> >>> discussion on some of the docs that most people
would never
>>> > > notice.".
>>> > > >> >>>
>>> > > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <
>>> rmetzger@apache.org>
>>> > > >> wrote:
>>> > > >> >>>
>>> > > >> >>>> I also like the proposal. I think its an
issue that Google
>>> Docs
>>> > > >> comments
>>> > > >> >>>> are not reflected within ASF infra. Therefore,
I'm +1 on
>>> > discussing
>>> > > >> the
>>> > > >> >>>> proposals on the mailing list.
>>> > > >> >>>>
>>> > > >> >>>> I agree that we need to clean up our wiki.
>>> > > >> >>>>
>>> > > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias
J. Sax <
>>> > mjsax@apache.org
>>> > > >
>>> > > >> >>>> wrote:
>>> > > >> >>>>
>>> > > >> >>>>> Just to point out one thing about Kafka
KIPs and using the
>>> > project
>>> > > >> >>>> wiki:
>>> > > >> >>>>>
>>> > > >> >>>>> The wiki contains the current state of
the proposal, while
>>> the
>>> > > >> >>>>> discussion is covered over the dev-mailing
list. IMHO, this
>>> > makes
>>> > > a
>>> > > >> lot
>>> > > >> >>>>> of sense, as people tend to follow the
mailing list but not
>>> wiki
>>> > > >> >>>>> changes. Furthermore, the mailing list
tracks the whole
>>> > discussion
>>> > > >> >>>>> history, while the proposal is kept in
a clean state and thus
>>> > > easy to
>>> > > >> >>>> read.
>>> > > >> >>>>>
>>> > > >> >>>>> -Matthias
>>> > > >> >>>>>
>>> > > >> >>>>>
>>> > > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek
wrote:
>>> > > >> >>>>>> Jip, that's why I referenced the
Kafka process which is
>>> also in
>>> > > >> their
>>> > > >> >>>>> wiki.
>>> > > >> >>>>>>
>>> > > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan
Ewen <sewen@apache.org>
>>> > > wrote:
>>> > > >> >>>>>>
>>> > > >> >>>>>>> Yes, big +1
>>> > > >> >>>>>>>
>>> > > >> >>>>>>> I had actually talked about the
same thing with some
>>> people as
>>> > > >> well.
>>> > > >> >>>>>>>
>>> > > >> >>>>>>> I am currently sketching a few
FLIPs for things, like
>>> > > improvements
>>> > > >> >>>> to
>>> > > >> >>>>> the
>>> > > >> >>>>>>> Yarn/Mesos/Kubernetes integration
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>> One thing we should do here is
to actually structure the
>>> wiki
>>> > a
>>> > > bit
>>> > > >> >>>> to
>>> > > >> >>>>> make
>>> > > >> >>>>>>> it easier to find information
and proposals.
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM,
Ufuk Celebi <
>>> uce@apache.org>
>>> > > >> wrote:
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>> Hey Aljoscha,
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> thanks for this proposal.
I've somehow missed it last
>>> week. I
>>> > > like
>>> > > >> >>>> the
>>> > > >> >>>>>>>> idea very much and agree
with your assessment about the
>>> > > problems
>>> > > >> >>>> with
>>> > > >> >>>>>>>> the Google Doc approach.
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> Regarding the process: I'm
also in favour of adopting it
>>> from
>>> > > >> >>>> Kafka. I
>>> > > >> >>>>>>>> would not expect any problems
with this, but we can post a
>>> > > quick
>>> > > >> >>>> note
>>> > > >> >>>>>>>> to their ML.
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> @Matthias: The name works
for me. ;-)
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> – Ufuk
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19
PM, Matthias J. Sax <
>>> > > >> >>>> mjsax@apache.org>
>>> > > >> >>>>>>>> wrote:
>>> > > >> >>>>>>>>> FLIP ?? Really? :D
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>> http://www.maya.tv/en/character/flip
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>> -Matthias
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>> On 06/28/2016 06:26 PM,
Aljoscha Krettek wrote:
>>> > > >> >>>>>>>>>> I'm proposing to
add a formal process for how we deal
>>> with
>>> > > >> >>>> (major)
>>> > > >> >>>>>>>>>> improvements to Flink
and design docs. This has been
>>> > > mentioned
>>> > > >> >>>>> several
>>> > > >> >>>>>>>>>> times recently but
we never took any decisive action to
>>> > > actually
>>> > > >> >>>>>>>> implement
>>> > > >> >>>>>>>>>> such a process so
here we go.
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>> Right now, we have
Jira issues and we sometimes we have
>>> > > design
>>> > > >> >>>> docs
>>> > > >> >>>>>>>> that we
>>> > > >> >>>>>>>>>> keep in Google Docs.
Jamie recently added links to those
>>> > > that he
>>> > > >> >>>>> could
>>> > > >> >>>>>>>> find
>>> > > >> >>>>>>>>>> on the mailing list
to the Flink wiki:
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>
>>> > > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>>> > > >> >>>>>>>> The
>>> > > >> >>>>>>>>>> problem with these
is that a) the comments on the Google
>>> > Docs
>>> > > >> >>>> are not
>>> > > >> >>>>>>>>>> reflected in Jira
and the mailing list. There has been
>>> some
>>> > > very
>>> > > >> >>>>>>> active
>>> > > >> >>>>>>>>>> discussion on some
of the docs that most people would
>>> never
>>> > > >> >>>> notice.
>>> > > >> >>>>>>> The
>>> > > >> >>>>>>>>>> community therefore
might seem less active than it
>>> actually
>>> > > is.
>>> > > >> >>>> b)
>>> > > >> >>>>> the
>>> > > >> >>>>>>>>>> documents are not
very discoverable, if we had a clearly
>>> > > defined
>>> > > >> >>>>> place
>>> > > >> >>>>>>>>>> where we put them
and also prominently link to this on
>>> the
>>> > > Flink
>>> > > >> >>>>>>>> homepage
>>> > > >> >>>>>>>>>> this would greatly
help people that try to find out
>>> about
>>> > > >> current
>>> > > >> >>>>>>>>>> developments.
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>> Kafka has a process
like this:
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>
>>> > > >> >>>>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>>> > > >> >>>>>>>> .
>>> > > >> >>>>>>>>>> They call it KIP,
for Kafka Improvement Proposal. We
>>> could
>>> > > >> either
>>> > > >> >>>>>>> adapt
>>> > > >> >>>>>>>>>> this for Flink or
come up with our own process. Doing
>>> the
>>> > > former
>>> > > >> >>>>> would
>>> > > >> >>>>>>>> save
>>> > > >> >>>>>>>>>> us a lot of time
and I don't think the Kafka community
>>> > would
>>> > > >> >>>> mind us
>>> > > >> >>>>>>>>>> copying their process.
The subject also hints at this,
>>> our
>>> > > >> >>>> process
>>> > > >> >>>>>>>> could be
>>> > > >> >>>>>>>>>> called FLIP, for
Flink Improvement Proposal.
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>> What do you think?
Feedback is highly welcome. :-)
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>> Cheers,
>>> > > >> >>>>>>>>>> Aljoscha
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>
>>> > > >> >>>>>
>>> > > >> >>>>>
>>> > > >> >>>>
>>> > > >> >>>
>>> > > >> >
>>> > > >>
>>> > > >>
>>> > >
>>> >
>>>
>>

Mime
View raw message