flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: [DISCUSS] Releasing Flink 1.1.0
Date Fri, 01 Jul 2016 17:14:21 GMT
Yet another one for the release: FLINK-4144
https://github.com/apache/flink/pull/2191

On Fri, Jul 1, 2016 at 5:18 PM, Maximilian Michels <mxm@apache.org> wrote:
> There is also FLINK-4141. We just found this during testing. PR is
> waiting to be merged here: https://github.com/apache/flink/pull/2190
>
> On Fri, Jul 1, 2016 at 3:26 PM, Maximilian Michels <mxm@apache.org> wrote:
>> FLINK-3904 is not Yarn related. Not pressing to fix for this release
>> and actually a bit tricky to fix. I've addressed the other issues and
>> merged all pending pull requests. Good to go from my side.
>>
>> On Fri, Jul 1, 2016 at 11:29 AM, Chesnay Schepler <chesnay@apache.org> wrote:
>>> There are also 2 fixes for Cassandra that  should be included:
>>> https://github.com/apache/flink/pull/2167
>>> https://github.com/apache/flink/pull/2183
>>>
>>> We should also include the documentation for the metrics stuff (hopefully
>>> merged today)
>>> https://github.com/apache/flink/pull/2158
>>>
>>> In regards to metrics: To add a counter metric a user currently has to call
>>> "counter(...)" on
>>> a MetricGroup. The point was raised in the documentation PR that we may want
>>> to give
>>> them a more descriptive name like "addCounter(...)".
>>>
>>> I would be in favor of changing them but would like others to weigh in on
>>> this. IMO we
>>> should nail this down before 1.1 .
>>>
>>> Regards,
>>> Chesnay
>>>
>>>
>>> On 30.06.2016 22:59, Greg Hogan wrote:
>>>>
>>>> It would be great if hash-based combine (FLINK-3477) could make it in to
>>>> be
>>>> tested for this release. We've seen impressive improvements in performance
>>>> (though, admittedly, some sort-based enhancements are yet to be worked
>>>> on).
>>>> This PR looks to be ripe.
>>>>
>>>> Also, as we tidy up a few things with Gelly and documentation, what is the
>>>> schedule for a feature freeze and creating a 1.1 branch off master?
>>>>
>>>> Thanks,
>>>> Greg
>>>>
>>>> On Mon, Jun 27, 2016 at 7:23 AM, Robert Metzger <rmetzger@apache.org>
>>>> wrote:
>>>>
>>>>> Sure Ufuk! Thanks a lot for taking care of the release management.
>>>>> I'll be on vacation in three weeks, for three weeks and I'm not sure
if
>>>>> we
>>>>> get the release done until then.
>>>>>
>>>>> On Mon, Jun 27, 2016 at 12:08 PM, Ufuk Celebi <uce@apache.org>
wrote:
>>>>>
>>>>>> I would like to do it if that's OK with you Robert. I would follow
>>>>>> your suggestion and wait a few days until the following important
>>>>>> fixes are in:
>>>>>> - Savepoint headers and proper disposal (FLINK-4067 and
>>>>>> https://github.com/apache/flink/pull/2083)
>>>>>> - Metrics (https://github.com/apache/flink/pull/2146)
>>>>>> - Table API time support (https://github.com/apache/flink/pull/2150)
>>>>>> - Kafka at-least-once Producer (
>>>>>
>>>>> https://github.com/apache/flink/pull/2108)
>>>>>>
>>>>>> - Cassandra connector fixes (https://github.com/apache/flink/pull/2163)
>>>>>> - YARN client fixes (FLINK-3675, FLINK-3904 @Max: is there something
>>>>>
>>>>> else?)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 23, 2016 at 1:33 PM, Robert Metzger <rmetzger@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>> it doesn't seem that there are volunteers for the RM, so I'll
probably
>>>>>
>>>>> do
>>>>>>>
>>>>>>> it.
>>>>>>>
>>>>>>> I try to do the first release candidate (mostly for testing)
next week
>>>>>>
>>>>>> (it
>>>>>>>
>>>>>>> depends on the JIRAs fixed by then)
>>>>>>>
>>>>>>> On Thu, Jun 16, 2016 at 10:56 PM, Henry Saputra <
>>>>>
>>>>> henry.saputra@gmail.com
>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks for the reply, @Max. I was not aware it was about
dynamic
>>>>>>
>>>>>> scaling,
>>>>>>>>
>>>>>>>> which I think also asked for YARN support.
>>>>>>>> I agree to list all related half merge JIRA for the ResourceManager.
>>>>>>>>
>>>>>>>> Looking forward for the Apache Mesos integration design for
sure =)
>>>>>>>>
>>>>>>>> - Henry
>>>>>>>>
>>>>>>>> On Thu, Jun 16, 2016 at 2:12 AM, Maximilian Michels <mxm@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Robert, hi Henry,
>>>>>>>>>
>>>>>>>>> +1 for a 1.1.0 release soon! We have enough new features
that
>>>>>
>>>>> justify
>>>>>>>>>
>>>>>>>>> a major release.
>>>>>>>>>
>>>>>>>>> @Henry We have plans to extend the ResourceManager to
interact with
>>>>>>>>> the Scheduler which will be a prerequisite for dynamic
scaling. I
>>>>>>>>> think this is out of scope for 1.1.0. The upcoming Mesos
integration
>>>>>>>>> won't require additional refactoring of the ResourceManager.
>>>>>
>>>>> Instead,
>>>>>>>>>
>>>>>>>>> we will create a new "Dispatcher" component that takes
care of
>>>>>>>>> bootstrapping the initial node with the JobManager/ResourceManager.
>>>>>>>>>  From there on, everything will be handled by the Mesos
>>>>>>>>> ResourceManager. I recently discussed this with Eron
(CC) who came
>>>>>
>>>>> up
>>>>>>>>>
>>>>>>>>> with this design and he plans to publish it to the mailing
list
>>>>>
>>>>> soon.
>>>>>>>>>
>>>>>>>>> How about listing relevant JIRA issues here? "Half Merged"
is kind
>>>>>
>>>>> of
>>>>>>>>>
>>>>>>>>> hard to get for people who are not involved in the different
>>>>>>>>> components.
>>>>>>>>>
>>>>>>>>> The Cassandra adapter seems like a pretty important thing
to have
>>>>>
>>>>> for
>>>>>>>>>
>>>>>>>>> the next release. In addition, I would like to merge
FLINK-3667 and
>>>>>>>>> FLINK-3937. Robert is doing a review at the moment :)
Those are a)
>>>>>>>>> refactoring of the command-line and client classes b)
adding
>>>>>>>>> capability to resume cluster programmatically.
>>>>>>>>>
>>>>>>>>> Then we should also have a look at any other critical/major
bugs
>>>>>>
>>>>>> listed
>>>>>>>>
>>>>>>>> in
>>>>>>>>>
>>>>>>>>> JIRA.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Max
>>>>>>>>>
>>>>>>>>> On Wed, Jun 15, 2016 at 10:50 PM, Henry Saputra <
>>>>>>
>>>>>> henry.saputra@gmail.com
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Robert,
>>>>>>>>>>
>>>>>>>>>> Thanks for staying the discussion.
>>>>>>>>>>
>>>>>>>>>> Do you know if there any open tasks for the Resource
Manager left?
>>>>>>>>>>
>>>>>>>>>> That is probably needed for Mesos integration?
>>>>>>>>>>
>>>>>>>>>> - Henry
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 15, 2016 at 12:55 PM, Robert Metzger
<
>>>>>>
>>>>>> rmetzger@apache.org>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Flink 1.0.0 was released early March, so three
months have passed
>>>>>>
>>>>>> and
>>>>>>>>
>>>>>>>> I
>>>>>>>>>>>
>>>>>>>>>>> think we should start discussing the scope of
the next major
>>>>>>
>>>>>> release
>>>>>>>>>>>
>>>>>>>>>>> (1.1.0).
>>>>>>>>>>>
>>>>>>>>>>>  From a high level point of view, we've added
the following new
>>>>>>>>
>>>>>>>> features:
>>>>>>>>>>>
>>>>>>>>>>> in master:
>>>>>>>>>>> - Table API Refactoring, SQL, StreamSQL
>>>>>>>>>>> - The metrics system
>>>>>>>>>>> - Kinesis Connector
>>>>>>>>>>> - Persistent file sources for streaming
>>>>>>>>>>>
>>>>>>>>>>> Half merged:
>>>>>>>>>>> - Resource manager refactoring
>>>>>>>>>>>
>>>>>>>>>>> Unmerged features:
>>>>>>>>>>> - Cassandra connector
>>>>>>>>>>> - Key groups ("rescaling from savepoints")
>>>>>>>>>>> - Queryable state
>>>>>>>>>>>
>>>>>>>>>>> I'm pretty sure I forgot many other features
/ pull requests,
>>>>>>
>>>>>> please
>>>>>>>>>
>>>>>>>>> post
>>>>>>>>>>>
>>>>>>>>>>> them to this thread. I'll collect them and create
a Wiki page out
>>>>>>
>>>>>> of
>>>>>>>>
>>>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>> Some immediate TODOs for us:
>>>>>>>>>>> - Which of the unmerged features are we going
to add to the
>>>>>>
>>>>>> release?
>>>>>>>>>>>
>>>>>>>>>>> - Which blockers do we need to address before
releasing?
>>>>>>>>>>> - Are there any volunteers for the release manager?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Robert
>>>>>>>>>>>
>>>

Mime
View raw message