flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: [VOTE] Release Apache Flink 0.10.0 (release-0.10.0-rc0)
Date Thu, 22 Oct 2015 09:53:00 GMT
Ah. Good catch! :) The job counter should not be limited by the size
of the job history. https://issues.apache.org/jira/browse/FLINK-2206
fixed that for the old web frontend but since we have a new one now...

On Thu, Oct 22, 2015 at 11:38 AM, Vasiliki Kalavri
<vasilikikalavri@gmail.com> wrote:
> Thanks!
> But shouldn't the "completed jobs" counter be updated regardless of the
> configuration value of how much history we store?
>
> On 22 October 2015 at 11:32, Stephan Ewen <sewen@apache.org> wrote:
>
>> @Vasia: We cannot keep infinite history currently, because the history is
>> kept in memory. Should improve that in the future...
>>
>> On Thu, Oct 22, 2015 at 11:31 AM, Maximilian Michels <mxm@apache.org>
>> wrote:
>>
>> > @Stephan: That's right, the detached mode is very useful for streaming
>> > programs. Let's see if we can merge Sachin's pull request to give more
>> > meaningful exceptions in case of user programs which are not
>> > compatible with the detached execution mode.
>> >
>> > @Vasia: That's a feature :) You can adjust the number of old jobs to
>> > be kept by setting 'jobmanager.web.history' (default is 5).
>> >
>> >
>> > On Thu, Oct 22, 2015 at 11:28 AM, Vasiliki Kalavri
>> > <vasilikikalavri@gmail.com> wrote:
>> > > Hi,
>> > >
>> > > I found an issue with the web interface. It only shows the last 5
>> > finished
>> > > jobs and the "Jobs Finished" counter also goes up to 5.
>> > > I found FLINK-2206, but it seems that this was fixed for the old
>> > interface
>> > > and broken again in the new one?
>> > > Shall I open another issue or is there something I need to configure in
>> > the
>> > > new web interface and I haven't?
>> > >
>> > > Thanks,
>> > > -V.
>> > >
>> > > On 22 October 2015 at 11:20, Stephan Ewen <sewen@apache.org> wrote:
>> > >
>> > >> I am onto FLINK-2800 and FLINK-2888
>> > >>
>> > >> I would not disable YARN detached mode, it is used quite a bit by
>> > streaming
>> > >> users and makes perfect sense for streaming jobs, which are always
>> > one-shot
>> > >> currently.
>> > >>
>> > >> On Thu, Oct 22, 2015 at 11:03 AM, Maximilian Michels <mxm@apache.org>
>> > >> wrote:
>> > >>
>> > >> > I had a gut feeling this wouldn't be the last RC :)
>> > >> >
>> > >> > I second Stephan, Ufuk, and Fabian. It's a good idea to start
>> testing
>> > the
>> > >> > release candidate. We might discover more issues on the way. In
the
>> > >> > meantime, let's fix FLINK-2763 and FLINK-2800 and push those to
the
>> > >> > release-0.10 branch.
>> > >> >
>> > >> > Eager execution calls are not supported by the detached YARN
>> execution
>> > >> > mode. The Flink standalone cluster mode doesn't have a detached
>> mode.
>> > >> > Sachin was working on throwing a proper exception for programs
that
>> > try
>> > >> to
>> > >> > submit a job which contains eager execution calls. In the course
of
>> > >> > throwing a proper exception, he also added detached execution
mode
>> > >> support
>> > >> > for the standalone cluster mode. The reason why eager execution
>> > doesn't
>> > >> > work with detached programs is that the client submits the first
>> Flink
>> > >> job
>> > >> > (call to execute/count/collect/print) to the cluster and then
>> returns
>> > an
>> > >> > empty ExecutionResult. This leads to error if the user program
tries
>> > to
>> > >> > access the ExecutionResult.
>> > >> >
>> > >> > Sachin's pull request hasn't been merged in time for the release.
I
>> > would
>> > >> > like Flink to support detached execution mode but I suggest to
>> disable
>> > >> > detached execution mode for YARN in this release. We can include
a
>> > proper
>> > >> > support for the next release.
>> > >> >
>> > >> > On Thu, Oct 22, 2015 at 9:50 AM, Sachin Goel <
>> > sachingoel0101@gmail.com>
>> > >> > wrote:
>> > >> >
>> > >> > > Not sure if it's a blocker, but the yarn detached mode is
faulty
>> for
>> > >> > > interactive programs with eager execution calls. The most
basic
>> > >> starting
>> > >> > > point for yarn, i.e. *bin/flink run -m yarn-cluster -yd -n
<>
>> > >> > > examples/Wordcount.jar* fails in a bad way.
>> > >> > >
>> > >> > >
>> > >> > > -- Sachin Goel
>> > >> > > Computer Science, IIT Delhi
>> > >> > > m. +91-9871457685
>> > >> > >
>> > >> > > On Thu, Oct 22, 2015 at 3:57 AM, fhueske <fhueske@gmail.com>
>> wrote:
>> > >> > >
>> > >> > > > +1 to that, Stephan.
>> > >> > > >
>> > >> > > > I can help with FLINK-2763 or FLINK-2800.
>> > >> > > >
>> > >> > > >
>> > >> > > > From: Stephan Ewen
>> > >> > > > Sent: Thursday, October 22, 2015 0:02
>> > >> > > > To: dev@flink.apache.org
>> > >> > > > Subject: Re: [VOTE] Release Apache Flink 0.10.0
>> > (release-0.10.0-rc0)
>> > >> > > >
>> > >> > > >
>> > >> > > > From my side 2888 is a valid blocker. Aljoscha also
found
>> another
>> > >> > blocker
>> > >> > > > bug, so this RC will need a few patches.
>> > >> > > >
>> > >> > > > I think for 2824 there was no consensus to what would
actually
>> be
>> > the
>> > >> > > > desired behavior, which makes it a bad candidate for
a release
>> > >> blocker.
>> > >> > > >
>> > >> > > > I would try and fix FLINK-2763 and FLINK-2800 if possible,
but
>> not
>> > >> > block
>> > >> > > > the release on that. They seem to be very corner case.
Good to
>> fix
>> > >> > them,
>> > >> > > > but not blockers. Too many people are on the 0.10 SNAPSHOT
right
>> > now
>> > >> > and
>> > >> > > > too many urgent fixes are in that people wait to be
available
>> in a
>> > >> > > release.
>> > >> > > >
>> > >> > > > How about we start testing anyways, because I would
expect us to
>> > find
>> > >> > > more
>> > >> > > > issues, and we save time if we do not create a new release
>> > candidate
>> > >> > for
>> > >> > > > each patch.
>> > >> > > >
>> > >> > > > On Wed, Oct 21, 2015 at 11:44 PM, Flavio Pompermaier
<
>> > >> > > pompermaier@okkam.it
>> > >> > > > >
>> > >> > > > wrote:
>> > >> > > >
>> > >> > > > > I would also point out that Flink-2763 and Flink-2800
could be
>> > >> worth
>> > >> > of
>> > >> > > > > further investigations before this release
>> > >> > > > >
>> > >> > > > > Best,
>> > >> > > > > Flavio
>> > >> > > > > On 21 Oct 2015 23:33, "Gyula Fóra" <gyfora@apache.org>
wrote:
>> > >> > > > >
>> > >> > > > > > Thanks Max for the effort, this is going to
be huge :)
>> > >> > > > > >
>> > >> > > > > > Unfortunately I have to say -1
>> > >> > > > > >
>> > >> > > > > > FLINK-2888 and FLINK-2824 are blockers from
my point of
>> view.
>> > >> > > > > >
>> > >> > > > > > Cheers,
>> > >> > > > > > Gyula
>> > >> > > > > >
>> > >> > > > > > Vasiliki Kalavri <vasilikikalavri@gmail.com>
ezt írta
>> > (időpont:
>> > >> > > 2015.
>> > >> > > > > okt.
>> > >> > > > > > 21., Sze, 20:07):
>> > >> > > > > >
>> > >> > > > > > > Awesome! Thanks Max :))
>> > >> > > > > > >
>> > >> > > > > > > I have a couple of questions:
>> > >> > > > > > > - what about the blocker issue (according
to the wiki)
>> > >> > FLINK-2747?
>> > >> > > > > > > - weren't we going to get rid of staging
altogether?
>> > >> > > > > > >
>> > >> > > > > > > Cheers,
>> > >> > > > > > > -V.
>> > >> > > > > > >
>> > >> > > > > > > On 21 October 2015 at 19:54, Stephan
Ewen <
>> sewen@apache.org
>> > >
>> > >> > > wrote:
>> > >> > > > > > >
>> > >> > > > > > > > Super, thanks Max!
>> > >> > > > > > > >
>> > >> > > > > > > > We should also bump the master to
the next version then,
>> > to
>> > >> > > > separate
>> > >> > > > > > what
>> > >> > > > > > > > goes into release fixes and what
goes into the next
>> > >> version...
>> > >> > > > > > > >
>> > >> > > > > > > > Is that going to be 1.0-SNAPSHOT?
;-) That is a separate
>> > >> > thread,
>> > >> > > I
>> > >> > > > > > > guess...
>> > >> > > > > > > >
>> > >> > > > > > > > On Wed, Oct 21, 2015 at 7:12 PM,
Maximilian Michels <
>> > >> > > > mxm@apache.org>
>> > >> > > > > > > > wrote:
>> > >> > > > > > > >
>> > >> > > > > > > > > Release candidates have to
be tested thoroughly.
>> > >> Therefore, I
>> > >> > > > would
>> > >> > > > > > > like
>> > >> > > > > > > > > everybody to take a look at
the release page in the
>> > wiki:
>> > >> > > > > > > > >
>> > >> > https://cwiki.apache.org/confluence/display/FLINK/0.10+Release
>> > >> > > > > > > > >
>> > >> > > > > > > > > I've compiled the checks into
a document. I would like
>> > >> > everyone
>> > >> > > > to
>> > >> > > > > > > assign
>> > >> > > > > > > > > one of the checks in the documents
to test the release
>> > >> > > candidate:
>> > >> > > > > > > > >
>> > >> > > > > > > > >
>> > >> > > > > > > >
>> > >> > > > > > >
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> https://docs.google.com/document/d/1TWCFj55xTyJjGYe8x9YEqmICgSvcexDPlbgP4CnLpLY/edit?usp=sharing
>> > >> > > > > > > > >
>> > >> > > > > > > > > On Wed, Oct 21, 2015 at 7:10
PM, Maximilian Michels <
>> > >> > > > > mxm@apache.org>
>> > >> > > > > > > > > wrote:
>> > >> > > > > > > > >
>> > >> > > > > > > > > > Dear community,
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > The past months we have
been working very hard to
>> push
>> > >> > > towards
>> > >> > > > > > 0.10.
>> > >> > > > > > > I
>> > >> > > > > > > > > > would like to propose
the first release candidate.
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > ===================================
>> > >> > > > > > > > > > Please vote on releasing
the following candidate as
>> > >> Apache
>> > >> > > > Flink
>> > >> > > > > > > > version
>> > >> > > > > > > > > > 0.10.0:
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > The commit to be voted
on:
>> > >> > > > > > > > > > b697064b71b97e51703caae13660038949d41631
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > Branch:
>> > >> > > > > > > > > > release-0.10.0-rc0 (see
>> > >> > > > > > > > > >
>> > >> > https://git1-us-west.apache.org/repos/asf/flink/?p=flink.git
>> > >> > > )
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > The release artifacts
to be voted on can be found
>> at:
>> > >> > > > > > > > > > http://people.apache.org/~mxm/flink-0.10.0-rc0/
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > Release artifacts are
signed with the key with
>> > >> fingerprint
>> > >> > > > > > C2909CBF:
>> > >> > > > > > > > > > http://www.apache.org/dist/flink/KEYS
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > The staging repository
for this release can be found
>> > at:
>> > >> > > > > > > > > >
>> > >> > > > > > >
>> > >> > > >
>> > >>
>> https://repository.apache.org/content/repositories/orgapacheflink-1047
>> > >> > > > > > > > > >
>> > >> > -------------------------------------------------------------
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > Please vote on releasing
this package as Apache
>> Flink
>> > >> > 0.10.0.
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > The vote is open for the
next 72 hours and passes
>> if a
>> > >> > > majority
>> > >> > > > > of
>> > >> > > > > > at
>> > >> > > > > > > > > > least three +1 PMC votes
are cast.
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > The vote ends on Monday
October 26, 2015.
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > [ ] +1 Release this package
as Apache Flink 0.10.0
>> > >> > > > > > > > > > [ ] -1 Do not release
this package because ...
>> > >> > > > > > > > > >
>> > >> > > > > > > > > > ===================================
>> > >> > > > > > > > > >
>> > >> > > > > > > > >
>> > >> > > > > > > >
>> > >> > > > > > >
>> > >> > > > > >
>> > >> > > > >
>> > >> > > >
>> > >> > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>>

Mime
View raw message