flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: [DISCUSS] Releasing Flink 1.1.0
Date Fri, 01 Jul 2016 09:29:17 GMT
There are also 2 fixes for Cassandra that  should be included:
https://github.com/apache/flink/pull/2167
https://github.com/apache/flink/pull/2183

We should also include the documentation for the metrics stuff 
(hopefully merged today)
https://github.com/apache/flink/pull/2158

In regards to metrics: To add a counter metric a user currently has to 
call "counter(...)" on
a MetricGroup. The point was raised in the documentation PR that we may 
want to give
them a more descriptive name like "addCounter(...)".

I would be in favor of changing them but would like others to weigh in 
on this. IMO we
should nail this down before 1.1 .

Regards,
Chesnay

On 30.06.2016 22:59, Greg Hogan wrote:
> It would be great if hash-based combine (FLINK-3477) could make it in to be
> tested for this release. We've seen impressive improvements in performance
> (though, admittedly, some sort-based enhancements are yet to be worked on).
> This PR looks to be ripe.
>
> Also, as we tidy up a few things with Gelly and documentation, what is the
> schedule for a feature freeze and creating a 1.1 branch off master?
>
> Thanks,
> Greg
>
> On Mon, Jun 27, 2016 at 7:23 AM, Robert Metzger <rmetzger@apache.org> wrote:
>
>> Sure Ufuk! Thanks a lot for taking care of the release management.
>> I'll be on vacation in three weeks, for three weeks and I'm not sure if we
>> get the release done until then.
>>
>> On Mon, Jun 27, 2016 at 12:08 PM, Ufuk Celebi <uce@apache.org> wrote:
>>
>>> I would like to do it if that's OK with you Robert. I would follow
>>> your suggestion and wait a few days until the following important
>>> fixes are in:
>>> - Savepoint headers and proper disposal (FLINK-4067 and
>>> https://github.com/apache/flink/pull/2083)
>>> - Metrics (https://github.com/apache/flink/pull/2146)
>>> - Table API time support (https://github.com/apache/flink/pull/2150)
>>> - Kafka at-least-once Producer (
>> https://github.com/apache/flink/pull/2108)
>>> - Cassandra connector fixes (https://github.com/apache/flink/pull/2163)
>>> - YARN client fixes (FLINK-3675, FLINK-3904 @Max: is there something
>> else?)
>>>
>>>
>>>
>>> On Thu, Jun 23, 2016 at 1:33 PM, Robert Metzger <rmetzger@apache.org>
>>> wrote:
>>>> Hi,
>>>> it doesn't seem that there are volunteers for the RM, so I'll probably
>> do
>>>> it.
>>>>
>>>> I try to do the first release candidate (mostly for testing) next week
>>> (it
>>>> depends on the JIRAs fixed by then)
>>>>
>>>> On Thu, Jun 16, 2016 at 10:56 PM, Henry Saputra <
>> henry.saputra@gmail.com
>>>> wrote:
>>>>
>>>>> Thanks for the reply, @Max. I was not aware it was about dynamic
>>> scaling,
>>>>> which I think also asked for YARN support.
>>>>> I agree to list all related half merge JIRA for the ResourceManager.
>>>>>
>>>>> Looking forward for the Apache Mesos integration design for sure =)
>>>>>
>>>>> - Henry
>>>>>
>>>>> On Thu, Jun 16, 2016 at 2:12 AM, Maximilian Michels <mxm@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Robert, hi Henry,
>>>>>>
>>>>>> +1 for a 1.1.0 release soon! We have enough new features that
>> justify
>>>>>> a major release.
>>>>>>
>>>>>> @Henry We have plans to extend the ResourceManager to interact with
>>>>>> the Scheduler which will be a prerequisite for dynamic scaling. I
>>>>>> think this is out of scope for 1.1.0. The upcoming Mesos integration
>>>>>> won't require additional refactoring of the ResourceManager.
>> Instead,
>>>>>> we will create a new "Dispatcher" component that takes care of
>>>>>> bootstrapping the initial node with the JobManager/ResourceManager.
>>>>>>  From there on, everything will be handled by the Mesos
>>>>>> ResourceManager. I recently discussed this with Eron (CC) who came
>> up
>>>>>> with this design and he plans to publish it to the mailing list
>> soon.
>>>>>> How about listing relevant JIRA issues here? "Half Merged" is kind
>> of
>>>>>> hard to get for people who are not involved in the different
>>>>>> components.
>>>>>>
>>>>>> The Cassandra adapter seems like a pretty important thing to have
>> for
>>>>>> the next release. In addition, I would like to merge FLINK-3667 and
>>>>>> FLINK-3937. Robert is doing a review at the moment :) Those are a)
>>>>>> refactoring of the command-line and client classes b) adding
>>>>>> capability to resume cluster programmatically.
>>>>>>
>>>>>> Then we should also have a look at any other critical/major bugs
>>> listed
>>>>> in
>>>>>> JIRA.
>>>>>>
>>>>>> Cheers,
>>>>>> Max
>>>>>>
>>>>>> On Wed, Jun 15, 2016 at 10:50 PM, Henry Saputra <
>>> henry.saputra@gmail.com
>>>>>> wrote:
>>>>>>> Hi Robert,
>>>>>>>
>>>>>>> Thanks for staying the discussion.
>>>>>>>
>>>>>>> Do you know if there any open tasks for the Resource Manager
left?
>>>>>>>
>>>>>>> That is probably needed for Mesos integration?
>>>>>>>
>>>>>>> - Henry
>>>>>>>
>>>>>>> On Wed, Jun 15, 2016 at 12:55 PM, Robert Metzger <
>>> rmetzger@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Flink 1.0.0 was released early March, so three months have
passed
>>> and
>>>>> I
>>>>>>>> think we should start discussing the scope of the next major
>>> release
>>>>>>>> (1.1.0).
>>>>>>>>
>>>>>>>>  From a high level point of view, we've added the following
new
>>>>> features:
>>>>>>>> in master:
>>>>>>>> - Table API Refactoring, SQL, StreamSQL
>>>>>>>> - The metrics system
>>>>>>>> - Kinesis Connector
>>>>>>>> - Persistent file sources for streaming
>>>>>>>>
>>>>>>>> Half merged:
>>>>>>>> - Resource manager refactoring
>>>>>>>>
>>>>>>>> Unmerged features:
>>>>>>>> - Cassandra connector
>>>>>>>> - Key groups ("rescaling from savepoints")
>>>>>>>> - Queryable state
>>>>>>>>
>>>>>>>> I'm pretty sure I forgot many other features / pull requests,
>>> please
>>>>>> post
>>>>>>>> them to this thread. I'll collect them and create a Wiki
page out
>>> of
>>>>> it.
>>>>>>>> Some immediate TODOs for us:
>>>>>>>> - Which of the unmerged features are we going to add to the
>>> release?
>>>>>>>> - Which blockers do we need to address before releasing?
>>>>>>>> - Are there any volunteers for the release manager?
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Robert
>>>>>>>>


Mime
View raw message