flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@apache.org>
Subject Re: Planning Flink release 0.7-incubating
Date Tue, 07 Oct 2014 15:20:59 GMT
So which PRs will be included in the candidate?

#141 POJOs
#144 Deprecate Record API
#136 Fixed example quickstart
#143(?) Hadoop Compat: Documentation + Hadoop function wrappers (includes
PR #131)

2014-10-07 16:26 GMT+02:00 Stephan Ewen <sewen@apache.org>:

> Great news, looking forward to seeing this in the master!
>
> On Tue, Oct 7, 2014 at 1:53 PM, Robert Metzger <rmetzger@apache.org>
> wrote:
>
> > As an update for everyone: My POJO feature is finished, including
> > documentation.
> > Aljoscha is currently adopting the Scala API to have support for (nested)
> > POJOs as well.
> > Once that is done, I'll merge everything and create a first candidate
> that
> > we can use for testing.
> >
> > On Tue, Sep 30, 2014 at 10:49 PM, Fabian Hueske <fhueske@apache.org>
> > wrote:
> >
> > > I just checked the "Run Example" quickstart and it needs a bit of work.
> > >
> > > 2014-09-30 22:41 GMT+02:00 Robert Metzger <rmetzger@apache.org>:
> > >
> > > > I'm working hard on getting the POJOs ready.
> > > >
> > > > We also should do a pass over our documentation, the quickstarts and
> > the
> > > > website to see if everything is in a good state (for example the
> > > > collection-based execution needs documentation as well). We should
> also
> > > > finally document the hadoop-input format wrappers (I think Timo is
> > > working
> > > > on a pull request for that).
> > > > This page mentions the LocalDistributedExecutor and contains some
> (most
> > > > likely outdated) scala code:
> > > >
> > >
> >
> http://flink.incubator.apache.org/docs/0.7-incubating/local_execution.html
> > > >
> > > > We also need to deprecate the old record api (
> > > > https://issues.apache.org/jira/browse/FLINK-1106).
> > > >
> > > > On Tue, Sep 30, 2014 at 9:17 PM, Stephan Ewen <sewen@apache.org>
> > wrote:
> > > >
> > > > > I think we are approaching ready state.
> > > > > Last issues are going in and we started working on dependencies and
> > > test
> > > > > platform diversity in order to make stabilizing phase.
> > > > >
> > > > > We should have an official feature freeze soon and fork the
> > 0.7-release
> > > > > branch. I personally vote to include the POJO support (I think
> Robert
> > > is
> > > > > sorting that one out and is close to completion), and I want to add
> > the
> > > > > collection-based execution (today or tomorrow). Till has the BLOB
> > > manager
> > > > > ready, which would be good to include (better support large
> libraries
> > > or
> > > > > fat jars).
> > > > >
> > > > > After that, I vote to freeze.
> > > > >
> > > > > On Tue, Sep 30, 2014 at 9:12 PM, Gyula Fora <gyfora@apache.org>
> > wrote:
> > > > >
> > > > > > Hey,
> > > > > >
> > > > > > So what is the current decision regarding the time of the
> upcoming
> > > > > release?
> > > > > >
> > > > > > As for the streaming component, we included all the features
we
> > > wanted,
> > > > > we
> > > > > > will start to test everything tomorrow, making sure that all
> works
> > as
> > > > > > intended.
> > > > > > We are also almost finished with cleaning up the connector
> > > dependencies
> > > > > > that Stephan pointed out, should be finished by tomorrow.
> > > > > >
> > > > > > Gyula
> > > > > >
> > > > > > On 26 Sep 2014, at 10:49, Fabian Hueske <fhueske@apache.org>
> > wrote:
> > > > > >
> > > > > > > +1 to manage this on JIRA (if possible)
> > > > > > >
> > > > > > > 2014-09-26 10:40 GMT+02:00 Aljoscha Krettek <
> aljoscha@apache.org
> > >:
> > > > > > >
> > > > > > >> Can we not manage this stuff on Jira?
> > > > > > >>
> > > > > > >> On Fri, Sep 26, 2014 at 10:16 AM, Stephan Ewen <
> > sewen@apache.org>
> > > > > > wrote:
> > > > > > >>> I personally like the idea of SOFT time-based feature
> freezes.
> > > > > > Otherwise,
> > > > > > >>> releases will get delayed again and again, because
of
> features
> > > that
> > > > > we
> > > > > > >> try
> > > > > > >>> to squeeze in.
> > > > > > >>>
> > > > > > >>> Having reached the freeze point already, we could
still
> include
> > > the
> > > > > > >>> features that are pending ready state in the next
days
> > > (streaming,
> > > > > blob
> > > > > > >>> Manager, POJOs), but otherwise head for a release
state.
> > > > > > >>>
> > > > > > >>> We had a mail listing the issues to include into
0.7, but a
> > wiki
> > > > page
> > > > > > >> would
> > > > > > >>> probably be better. In that sense, we could start
collecting
> > the
> > > > > issues
> > > > > > >> for
> > > > > > >>> the next release from now on.
> > > > > > >>> Am 26.09.2014 09:17 schrieb "Daniel Warneke" <
> > warneke@apache.org
> > > >:
> > > > > > >>>
> > > > > > >>>> Hi,
> > > > > > >>>>
> > > > > > >>>> I like Fabian's idea. Is there a wiki page
(or something
> > > similar)
> > > > > > where
> > > > > > >> we
> > > > > > >>>> can collect the proposed JIRAs?
> > > > > > >>>>
> > > > > > >>>> Best regards,
> > > > > > >>>>
> > > > > > >>>>    Daniel
> > > > > > >>>>
> > > > > > >>>> Am 24.09.2014 23:03, schrieb Fabian Hueske:
> > > > > > >>>>
> > > > > > >>>>> I agree, a hard feature stop deadline might
not be the best
> > > > > practice.
> > > > > > >>>>>
> > > > > > >>>>> How about the following procedure:
> > > > > > >>>>> We decide two (or three) weeks before a
targeted release
> date
> > > > about
> > > > > > >> which
> > > > > > >>>>> JIRAs to include. JIRAs that are selected
for a release
> > should
> > > be
> > > > > > >>>>> completed
> > > > > > >>>>> or really close to completion (via progress
estimates in
> > JIRA).
> > > > > > >>>>> After we decided which JIRAs to include
in a release, we
> can
> > > use
> > > > > JIRA
> > > > > > >> to
> > > > > > >>>>> track the progress and dedicate another
week exclusively
> for
> > > > > testing
> > > > > > >> after
> > > > > > >>>>> the last feature was completed.
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>> 2014-09-24 19:10 GMT+02:00 Márton Balassi
<
> > > > > balassi.marton@gmail.com
> > > > > > >:
> > > > > > >>>>>
> > > > > > >>>>> As for the streaming team we're also getting
ready for the
> > > > release,
> > > > > > >> but a
> > > > > > >>>>>> couple of days will be needed to finish
the features that
> we
> > > > would
> > > > > > >> like
> > > > > > >>>>>> to
> > > > > > >>>>>> include.
> > > > > > >>>>>>
> > > > > > >>>>>>    - A little work is still needed
for reduce operations
> and
> > > > > > >>>>>>    groups/connected streams (any comment
on Gyula's recent
> > > > e-mail
> > > > > is
> > > > > > >>>>>> really
> > > > > > >>>>>>    appreciated :) )
> > > > > > >>>>>>    - The examples are being updated
to match the standard,
> > > check
> > > > > out
> > > > > > >> the
> > > > > > >>>>>>    WordCount. (
> > > > > > >>>>>>
> > > > > > >>>>>> https://github.com/mbalassi/incubator-flink/blob/
> > > > > > >>>>>> streaming-new/flink-addons/flink-streaming/flink-
> > > > > > >>>>>> streaming-examples/src/main/java/org/apache/flink/
> > > > > > >>>>>> streaming/examples/wordcount/WordCount.java
> > > > > > >>>>>> )
> > > > > > >>>>>>    Hopefully it gives you some deja
vu. :)
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> On Wed, Sep 24, 2014 at 6:53 PM, Ufuk
Celebi <
> > uce@apache.org>
> > > > > > wrote:
> > > > > > >>>>>>
> > > > > > >>>>>> On 24 Sep 2014, at 18:37, Robert Metzger
<
> > rmetzger@apache.org
> > > >
> > > > > > >> wrote:
> > > > > > >>>>>>>
> > > > > > >>>>>>> Hey guys,
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> exactly 3 weeks ago, we discussed
to do a feature freeze
> > for
> > > > the
> > > > > > >>>>>>>> 0.7-incubating release today.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> From our initial feature list:
> > > > > > >>>>>>>> - *Flink Streaming* "Beta Preview".
I would suggest to
> > ship
> > > > the
> > > > > > >>>>>>>>
> > > > > > >>>>>>> streaming,
> > > > > > >>>>>>>
> > > > > > >>>>>>>> but clearly mark it as a preview
in the documentation.
> > > > > > >>>>>>>> -* Java API Pojo improvements*:
Code generation, key
> > > selection
> > > > > > >> using a
> > > > > > >>>>>>>> string-expression:
> > > > > > https://issues.apache.org/jira/browse/FLINK-1032
> > > > > > >>>>>>>>  - *Reworked Scala API*. Bring
the Scala API in sync
> with
> > > the
> > > > > > >> latest
> > > > > > >>>>>>>> developments in the Java API:
> > > > > > >>>>>>>> https://issues.apache.org/jira/browse/FLINK-641
> > > > > > >>>>>>>>  -* Akka-based RPC service*:
> > > > > > >>>>>>>> https://issues.apache.org/jira/browse/FLINK-1019
> > > > > > >>>>>>>>  - *Kryo-based serialization*.
This feature has been
> > > requested
> > > > > by
> > > > > > >> many
> > > > > > >>>>>>>> users. Mostly because they
wanted to use Collections
> > inside
> > > > > POJOs:
> > > > > > >>>>>>>> https://issues.apache.org/jira/browse/FLINK-610
> > > > > > >>>>>>>> - Rework JobManager internals
to support incremental
> > program
> > > > > > >> rollout &
> > > > > > >>>>>>>> execution
> > > > > > >>>>>>>> - First parts of dynamic memory
assignments
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> The following features are
in the master, as of today:
> > > > > > >>>>>>>> - *Flink Streaming*
> > > > > > >>>>>>>> - *Reworked Scala API*
> > > > > > >>>>>>>> -* New Scheduler*
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> We certainly need some days
to test everything until we
> > can
> > > > > start
> > > > > > >> the
> > > > > > >>>>>>>>
> > > > > > >>>>>>> vote.
> > > > > > >>>>>>>
> > > > > > >>>>>>>> Based on our experience with
the last major release, I
> > would
> > > > > > really
> > > > > > >>>>>>>>
> > > > > > >>>>>>> like
> > > > > > >>>>>>
> > > > > > >>>>>>> to
> > > > > > >>>>>>>
> > > > > > >>>>>>>> do the testing and bugfixing
BEFORE the first release
> > > > candidate.
> > > > > > For
> > > > > > >>>>>>>>
> > > > > > >>>>>>> the
> > > > > > >>>>>>
> > > > > > >>>>>>> 0.6-incubating release, we had
6  candidates)
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> How do you guys feel about
this? Should we wait a few
> more
> > > > days
> > > > > > for
> > > > > > >> the
> > > > > > >>>>>>>> release so that a few more
features make it into the
> > > release?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> I'm undecided on this. On the
one hand, its really nice
> to
> > > > > release
> > > > > > >> on a
> > > > > > >>>>>>>> regular schedule, but it also
eats up some time and
> causes
> > > > > > overhead
> > > > > > >>>>>>>> (different branches etc.).
> > > > > > >>>>>>>> I would really like to have
the Java API Pojo
> improvements
> > > in
> > > > > the
> > > > > > >>>>>>>>
> > > > > > >>>>>>> release.
> > > > > > >>>>>>>
> > > > > > >>>>>>>> I think I can finish it until
end of this week.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Opinions?
> > > > > > >>>>>>>>
> > > > > > >>>>>>> I agree that the finished features
(especially the Scala
> > API)
> > > > are
> > > > > > >> nice
> > > > > > >>>>>>>
> > > > > > >>>>>> for
> > > > > > >>>>>>
> > > > > > >>>>>>> a new release, but still I would
like to wait a few more
> > > days.
> > > > > > >>>>>>>
> > > > > > >>>>>>> Some of the missing features are
on the brink of being
> > > finished
> > > > > > (e.g.
> > > > > > >>>>>>> the
> > > > > > >>>>>>> Pojo improvements). I wouldn't
want to invest a week in
> bug
> > > > > fixing
> > > > > > >> and
> > > > > > >>>>>>> doing the release vote, when the
new features are likely
> to
> > > be
> > > > > > >> finished
> > > > > > >>>>>>> just a few days afterwards. And
the upcoming features
> will
> > > > > > >> definitely be
> > > > > > >>>>>>> worth a release, so users can work
with them. ;)
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message