flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: [VOTE] Release Apache Flink 0.10.0 (release-0.10.0-rc0)
Date Thu, 22 Oct 2015 10:19:14 GMT
We still have this issue as blocker https://issues.apache.org/jira/browse/FLINK-2747

I don’t see a fix for this yet, however.

> On 22 Oct 2015, at 11:53, Maximilian Michels <mxm@apache.org> wrote:
> 
> Ah. Good catch! :) The job counter should not be limited by the size
> of the job history. https://issues.apache.org/jira/browse/FLINK-2206
> fixed that for the old web frontend but since we have a new one now...
> 
> On Thu, Oct 22, 2015 at 11:38 AM, Vasiliki Kalavri
> <vasilikikalavri@gmail.com> wrote:
>> Thanks!
>> But shouldn't the "completed jobs" counter be updated regardless of the
>> configuration value of how much history we store?
>> 
>> On 22 October 2015 at 11:32, Stephan Ewen <sewen@apache.org> wrote:
>> 
>>> @Vasia: We cannot keep infinite history currently, because the history is
>>> kept in memory. Should improve that in the future...
>>> 
>>> On Thu, Oct 22, 2015 at 11:31 AM, Maximilian Michels <mxm@apache.org>
>>> wrote:
>>> 
>>>> @Stephan: That's right, the detached mode is very useful for streaming
>>>> programs. Let's see if we can merge Sachin's pull request to give more
>>>> meaningful exceptions in case of user programs which are not
>>>> compatible with the detached execution mode.
>>>> 
>>>> @Vasia: That's a feature :) You can adjust the number of old jobs to
>>>> be kept by setting 'jobmanager.web.history' (default is 5).
>>>> 
>>>> 
>>>> On Thu, Oct 22, 2015 at 11:28 AM, Vasiliki Kalavri
>>>> <vasilikikalavri@gmail.com> wrote:
>>>>> Hi,
>>>>> 
>>>>> I found an issue with the web interface. It only shows the last 5
>>>> finished
>>>>> jobs and the "Jobs Finished" counter also goes up to 5.
>>>>> I found FLINK-2206, but it seems that this was fixed for the old
>>>> interface
>>>>> and broken again in the new one?
>>>>> Shall I open another issue or is there something I need to configure
in
>>>> the
>>>>> new web interface and I haven't?
>>>>> 
>>>>> Thanks,
>>>>> -V.
>>>>> 
>>>>> On 22 October 2015 at 11:20, Stephan Ewen <sewen@apache.org> wrote:
>>>>> 
>>>>>> I am onto FLINK-2800 and FLINK-2888
>>>>>> 
>>>>>> I would not disable YARN detached mode, it is used quite a bit by
>>>> streaming
>>>>>> users and makes perfect sense for streaming jobs, which are always
>>>> one-shot
>>>>>> currently.
>>>>>> 
>>>>>> On Thu, Oct 22, 2015 at 11:03 AM, Maximilian Michels <mxm@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>>> I had a gut feeling this wouldn't be the last RC :)
>>>>>>> 
>>>>>>> I second Stephan, Ufuk, and Fabian. It's a good idea to start
>>> testing
>>>> the
>>>>>>> release candidate. We might discover more issues on the way.
In the
>>>>>>> meantime, let's fix FLINK-2763 and FLINK-2800 and push those
to the
>>>>>>> release-0.10 branch.
>>>>>>> 
>>>>>>> Eager execution calls are not supported by the detached YARN
>>> execution
>>>>>>> mode. The Flink standalone cluster mode doesn't have a detached
>>> mode.
>>>>>>> Sachin was working on throwing a proper exception for programs
that
>>>> try
>>>>>> to
>>>>>>> submit a job which contains eager execution calls. In the course
of
>>>>>>> throwing a proper exception, he also added detached execution
mode
>>>>>> support
>>>>>>> for the standalone cluster mode. The reason why eager execution
>>>> doesn't
>>>>>>> work with detached programs is that the client submits the first
>>> Flink
>>>>>> job
>>>>>>> (call to execute/count/collect/print) to the cluster and then
>>> returns
>>>> an
>>>>>>> empty ExecutionResult. This leads to error if the user program
tries
>>>> to
>>>>>>> access the ExecutionResult.
>>>>>>> 
>>>>>>> Sachin's pull request hasn't been merged in time for the release.
I
>>>> would
>>>>>>> like Flink to support detached execution mode but I suggest to
>>> disable
>>>>>>> detached execution mode for YARN in this release. We can include
a
>>>> proper
>>>>>>> support for the next release.
>>>>>>> 
>>>>>>> On Thu, Oct 22, 2015 at 9:50 AM, Sachin Goel <
>>>> sachingoel0101@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Not sure if it's a blocker, but the yarn detached mode is
faulty
>>> for
>>>>>>>> interactive programs with eager execution calls. The most
basic
>>>>>> starting
>>>>>>>> point for yarn, i.e. *bin/flink run -m yarn-cluster -yd -n
<>
>>>>>>>> examples/Wordcount.jar* fails in a bad way.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> -- Sachin Goel
>>>>>>>> Computer Science, IIT Delhi
>>>>>>>> m. +91-9871457685
>>>>>>>> 
>>>>>>>> On Thu, Oct 22, 2015 at 3:57 AM, fhueske <fhueske@gmail.com>
>>> wrote:
>>>>>>>> 
>>>>>>>>> +1 to that, Stephan.
>>>>>>>>> 
>>>>>>>>> I can help with FLINK-2763 or FLINK-2800.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> From: Stephan Ewen
>>>>>>>>> Sent: Thursday, October 22, 2015 0:02
>>>>>>>>> To: dev@flink.apache.org
>>>>>>>>> Subject: Re: [VOTE] Release Apache Flink 0.10.0
>>>> (release-0.10.0-rc0)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> From my side 2888 is a valid blocker. Aljoscha also found
>>> another
>>>>>>> blocker
>>>>>>>>> bug, so this RC will need a few patches.
>>>>>>>>> 
>>>>>>>>> I think for 2824 there was no consensus to what would
actually
>>> be
>>>> the
>>>>>>>>> desired behavior, which makes it a bad candidate for
a release
>>>>>> blocker.
>>>>>>>>> 
>>>>>>>>> I would try and fix FLINK-2763 and FLINK-2800 if possible,
but
>>> not
>>>>>>> block
>>>>>>>>> the release on that. They seem to be very corner case.
Good to
>>> fix
>>>>>>> them,
>>>>>>>>> but not blockers. Too many people are on the 0.10 SNAPSHOT
right
>>>> now
>>>>>>> and
>>>>>>>>> too many urgent fixes are in that people wait to be available
>>> in a
>>>>>>>> release.
>>>>>>>>> 
>>>>>>>>> How about we start testing anyways, because I would expect
us to
>>>> find
>>>>>>>> more
>>>>>>>>> issues, and we save time if we do not create a new release
>>>> candidate
>>>>>>> for
>>>>>>>>> each patch.
>>>>>>>>> 
>>>>>>>>> On Wed, Oct 21, 2015 at 11:44 PM, Flavio Pompermaier
<
>>>>>>>> pompermaier@okkam.it
>>>>>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> I would also point out that Flink-2763 and Flink-2800
could be
>>>>>> worth
>>>>>>> of
>>>>>>>>>> further investigations before this release
>>>>>>>>>> 
>>>>>>>>>> Best,
>>>>>>>>>> Flavio
>>>>>>>>>> On 21 Oct 2015 23:33, "Gyula Fóra" <gyfora@apache.org>
wrote:
>>>>>>>>>> 
>>>>>>>>>>> Thanks Max for the effort, this is going to be
huge :)
>>>>>>>>>>> 
>>>>>>>>>>> Unfortunately I have to say -1
>>>>>>>>>>> 
>>>>>>>>>>> FLINK-2888 and FLINK-2824 are blockers from my
point of
>>> view.
>>>>>>>>>>> 
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Gyula
>>>>>>>>>>> 
>>>>>>>>>>> Vasiliki Kalavri <vasilikikalavri@gmail.com>
ezt írta
>>>> (időpont:
>>>>>>>> 2015.
>>>>>>>>>> okt.
>>>>>>>>>>> 21., Sze, 20:07):
>>>>>>>>>>> 
>>>>>>>>>>>> Awesome! Thanks Max :))
>>>>>>>>>>>> 
>>>>>>>>>>>> I have a couple of questions:
>>>>>>>>>>>> - what about the blocker issue (according
to the wiki)
>>>>>>> FLINK-2747?
>>>>>>>>>>>> - weren't we going to get rid of staging
altogether?
>>>>>>>>>>>> 
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> -V.
>>>>>>>>>>>> 
>>>>>>>>>>>> On 21 October 2015 at 19:54, Stephan Ewen
<
>>> sewen@apache.org
>>>>> 
>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Super, thanks Max!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> We should also bump the master to the
next version then,
>>>> to
>>>>>>>>> separate
>>>>>>>>>>> what
>>>>>>>>>>>>> goes into release fixes and what goes
into the next
>>>>>> version...
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Is that going to be 1.0-SNAPSHOT? ;-)
That is a separate
>>>>>>> thread,
>>>>>>>> I
>>>>>>>>>>>> guess...
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, Oct 21, 2015 at 7:12 PM, Maximilian
Michels <
>>>>>>>>> mxm@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Release candidates have to be tested
thoroughly.
>>>>>> Therefore, I
>>>>>>>>> would
>>>>>>>>>>>> like
>>>>>>>>>>>>>> everybody to take a look at the release
page in the
>>>> wiki:
>>>>>>>>>>>>>> 
>>>>>>> https://cwiki.apache.org/confluence/display/FLINK/0.10+Release
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I've compiled the checks into a document.
I would like
>>>>>>> everyone
>>>>>>>>> to
>>>>>>>>>>>> assign
>>>>>>>>>>>>>> one of the checks in the documents
to test the release
>>>>>>>> candidate:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>> https://docs.google.com/document/d/1TWCFj55xTyJjGYe8x9YEqmICgSvcexDPlbgP4CnLpLY/edit?usp=sharing
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Oct 21, 2015 at 7:10 PM,
Maximilian Michels <
>>>>>>>>>> mxm@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Dear community,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The past months we have been
working very hard to
>>> push
>>>>>>>> towards
>>>>>>>>>>> 0.10.
>>>>>>>>>>>> I
>>>>>>>>>>>>>>> would like to propose the first
release candidate.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> ===================================
>>>>>>>>>>>>>>> Please vote on releasing the
following candidate as
>>>>>> Apache
>>>>>>>>> Flink
>>>>>>>>>>>>> version
>>>>>>>>>>>>>>> 0.10.0:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>> b697064b71b97e51703caae13660038949d41631
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>> release-0.10.0-rc0 (see
>>>>>>>>>>>>>>> 
>>>>>>> https://git1-us-west.apache.org/repos/asf/flink/?p=flink.git
>>>>>>>> )
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The release artifacts to be voted
on can be found
>>> at:
>>>>>>>>>>>>>>> http://people.apache.org/~mxm/flink-0.10.0-rc0/
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Release artifacts are signed
with the key with
>>>>>> fingerprint
>>>>>>>>>>> C2909CBF:
>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The staging repository for this
release can be found
>>>> at:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>> 
>>> https://repository.apache.org/content/repositories/orgapacheflink-1047
>>>>>>>>>>>>>>> 
>>>>>>> -------------------------------------------------------------
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Please vote on releasing this
package as Apache
>>> Flink
>>>>>>> 0.10.0.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The vote is open for the next
72 hours and passes
>>> if a
>>>>>>>> majority
>>>>>>>>>> of
>>>>>>>>>>> at
>>>>>>>>>>>>>>> least three +1 PMC votes are
cast.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The vote ends on Monday October
26, 2015.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> [ ] +1 Release this package as
Apache Flink 0.10.0
>>>>>>>>>>>>>>> [ ] -1 Do not release this package
because ...
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> ===================================
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>> 


Mime
View raw message