flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: [DISCUSS] Feature Freeze
Date Wed, 03 May 2017 09:42:26 GMT
This is the list of blocking issues for Flink 1.3: https://issues.apache.org/jira/browse/FLINK-5740?jql=project%20%3D%20FLINK%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.3.0
<https://issues.apache.org/jira/browse/FLINK-5740?jql=project%20=%20FLINK%20AND%20status%20in%20(Open,%20%22In%20Progress%22,%20Reopened)%20AND%20priority%20=%20Blocker%20AND%20fixVersion%20=%201.3.0>

Could everyone please update the state of the issues mentioned in this thread to “blocking”
and set the version to “1.3.0” if they feel that this is appropriate. Otherwise it is
very hard for the release manager to get an overview over the progress of the release. With
an up-to-date list it’s also possible to triage issues and discuss on the issue whether
it really should be a release blocker.

Best,
Aljoscha
> On 2. May 2017, at 22:29, Eron Wright <eronwright@gmail.com> wrote:
> 
> Robert, I'd like to see FLINK-5974
> <https://issues.apache.org/jira/browse/FLINK-5974> (Mesos DNS support)
> added to the list of important issues.  A PR is ready.
> 
> On Tue, May 2, 2017 at 4:30 AM, Kostas Kloudas <k.kloudas@data-artisans.com>
> wrote:
> 
>> The only thing that I want to add in the features to be added for 1.3
>> is the NOT pattern for the CEP library.
>> 
>> There is an open PR here: https://github.com/apache/flink/pull/3798 <
>> https://github.com/apache/flink/pull/3798>
>> which is not far from getting in.
>> 
>> Kostas
>> 
>>> On May 2, 2017, at 12:10 PM, Robert Metzger <rmetzger@apache.org> wrote:
>>> 
>>> Thanks a lot Ufuk for starting the discussion regarding the 1.3 feature
>>> freeze.
>>> 
>>> I didn't feature freeze yesterday (Monday) because it was a public
>> holiday
>>> here in Germany.
>>> 
>>> I haven't made up my mind whether to do the feature freeze today or not.
>>> Many important features seem to be close to completion.
>>> I don't think we can pick features and wait with the release until they
>> are
>>> finished. The only thing I can imagine we could do is extend the
>> deadline a
>>> little bit to give people more time to complete.
>>> But I'm strictly against multiple extensions.
>>> 
>>> To make the discussion a bit easier, I've collected all the JIRAs
>> mentioned
>>> in this thread. I decided to group them into "really important" and
>>> "important". I hope nobody is offended by my selection and I'm very open
>> to
>>> discuss it.
>>> It doesn't really matter anyways because we will not block the release on
>>> the completion of certain features.
>>> 
>>> 
>>> Completed Features for 1.3
>>> - TODO!
>>> 
>>> Blockers:
>>> - TODO!
>>> 
>>> 
>>> Really important (open):
>>> - FLINK-6364: Implement incremental checkpointing in RocksDBStateBackend
>>> (pending PR)
>>> - FLINK-5906: Add support to register user defined aggregates in
>>> TableEnvironment (no PR)
>>> - FLINK-6047: Add support for Retraction in Table API / SQL (depends on
>>> FLINK-6093)
>>> - FLINK-6093: Implement and turn on retraction for table sink (pending
>> PR)
>>> - FLINK-6334: Refactoring UDTF interface (pending PR, under review)
>>> - FLINK-5998: Un-fat Hadoop from Flink fat jar (pending PR, under review)
>>> - FLINK-4545: Flink automatically manages TM network buffer (pending PR,
>>> under review)
>>> - FLINK-6178: Allow upgrades to state serializers (pending PR, no review
>> )
>>> 
>>> 
>>> Really important (closed):
>>> - FLINK-5892: Recover job state at the granularity of operator (merged)
>>> 
>>> 
>>> 
>>> Important (open):
>>> - FLINK-6013: Add Datadog HTTP metrics reporter (pending PR, under
>> review)
>>> - FLINK-6337: Remove the buffer provider from
>> PartitionRequestServerHandler
>>> (pending PR, under review)
>>> - FLINK-6033: Support UNNEST query in the stream SQL API (no PR)
>>> - FLINK-6335: Parse DISTINCT over grouped window in stream SQL (pending
>> PR)
>>> - FLINK-6373: Add runtime support for distinct aggregation over grouped
>>> windows (pending PR, under review)
>>> - FLINK-6281: Create TableSink for JDBC (pending PR, under review)
>>> - FLINK-6225: Support Row Stream for CassandraSink (pending PR, under
>>> review)
>>> - FLINK-6196: Support dynamic schema in Table Function (pending PR, under
>>> review)
>>> - FLINK-4022: Partition and topic discovery for FlinkKafkaConsumer
>> (pending
>>> PR, no review)
>>> - FLINK-4821: Implement rescalable non-partitioned state for Kinesis
>>> Connector (pending PR, under review)
>>> Important (closed):
>>> - FLINK-6377: Support map types in the Table / SQL API (merged)
>>> - FLINK-6398: RowSerializer's duplicate should always return a new
>> instance
>>> (merged)
>>> 
>>> 
>>> I'll now collect the list of finished features in 1.3 and the blockers
>> and
>>> update above list.
>>> 
>>> Stephan's email just came in, and I like the idea of freezing the feature
>>> set now but extending the deadline to Friday for the branching. (Friday
>> 3pm
>>> CEST)
>>> 
>>> Does everybody agree to that proposal?
>>> 
>>> 
>>> On Tue, May 2, 2017 at 12:05 PM, Stephan Ewen <sewen@apache.org> wrote:
>>> 
>>>> Thanks all for the lively discussion about the feature freeze and how to
>>>> proceed.
>>>> Because we committed to a time-based release schedule, we should not
>> break
>>>> the feature freeze too badly, or we would just disable the
>>>> time-based-release-policy at the very first time it would trigger.
>>>> 
>>>> Here are a few thoughts about what we can do:
>>>> 
>>>> - First of all, please note that feature freeze does not mean bug fix
>>>> freeze. Important bug fixes can and should go in as part of the 1.3.0
>>>> release testing period
>>>> 
>>>> - We should probably not add brand new features to the list at this
>> point
>>>> (that would most likely break the release schedule completely)
>>>> 
>>>> - I think there is a case to include some features which are partially
>>>> (but not fully) in master already, or exist basically completely as Pull
>>>> Requests at this time.
>>>> 
>>>> How about we do *freeze the set of features* for the release today, but
>>>> *leave
>>>> a grace period to get the complete-but-not-merged features in until the
>> end
>>>> of this week*?
>>>> That would mean forking the release-1.3 branch end of this week.
>>>> 
>>>> 
>>>> We should try and agree on which features fall under that list, and make
>>>> sure each one has a committer committed to merging them in the next
>> days.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Tue, May 2, 2017 at 11:01 AM, Tzu-Li (Gordon) Tai <
>> tzulitai@apache.org>
>>>> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> I have 2 parts to discuss about delaying the feature freeze for a bit:
>>>>> 
>>>>> 1. Allow upgrades to state serializers (FLINK-6178)
>>>>> The feature is split up into 2 PRs, one of which is already open and
>> the
>>>>> second one coming up very soon.
>>>>> PR #1 reconfigurable TypeSerializers: https://
>>>>> github.com/apache/flink/pull/3804
>>>>> PR #2 (will open soon) integrate serializer reconfiguration into state
>>>>> restore flow (FLINK-6425)
>>>>> 
>>>>> 2. Major new functionality for the Kafka and Kinesis consumers.
>>>>> Kafka - partition & topic regex discovery (PR -
>>>> https://github.com/apache/
>>>>> flink/pull/3746)
>>>>> Kinesis - rescalable Kinesis consumer (PR - https://github.com/apache/
>>>>> flink/pull/3001)
>>>>> 
>>>>> All above mentioned features are not yet merged to master. I think an
>>>>> additional few days to the end of this week for the feature freeze
>> should
>>>>> make this possible.
>>>>> 
>>>>> Cheers,
>>>>> Gordon
>>>>> 
>>>>> On 29 April 2017 at 4:31:17 AM, Chesnay Schepler (chesnay@apache.org)
>>>>> wrote:
>>>>> 
>>>>> FLINK-5892 has been merged.
>>>>> 
>>>>> For FLINK-4545 (replacing numNetworkBuffer parameter) a PR is also
>> still
>>>>> open and could use a second pair of eyes.
>>>>> 
>>>>> On 28.04.2017 17:03, Kurt Young wrote:
>>>>>> Hi Flavio,
>>>>>> 
>>>>>> I have also fix the issue in 1.2 branch, but the next release will
be
>>>>> 1.2.2
>>>>>> 
>>>>>> Best,
>>>>>> Kurt
>>>>>> 
>>>>>> On Fri, Apr 28, 2017 at 11:01 PM, Ted Yu <yuzhihong@gmail.com>
wrote:
>>>>>> 
>>>>>>> Flavio:
>>>>>>> Have you seen this (w.r.t. 1.2.1) ?
>>>>>>> 
>>>>>>> http://search-hadoop.com/m/Flink/VkLeQejxLg24Lk0D1?subj=+
>>>>>>> RESULT+VOTE+Release+Apache+Flink+1+2+1+RC2+
>>>>>>> 
>>>>>>> On Fri, Apr 28, 2017 at 5:07 AM, Flavio Pompermaier <
>>>>> pompermaier@okkam.it>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Any chance to cherry-pick this also into 1.2.1? We're usign
Flink
>>>> 1.2.0
>>>>>>> in
>>>>>>>> production and maybe an upgrade to 1.2.1 would be a safer
option in
>>>> the
>>>>>>>> short term..
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> Flavio
>>>>>>>> 
>>>>>>>> On Fri, Apr 28, 2017 at 2:00 PM, Aljoscha Krettek <
>>>> aljoscha@apache.org
>>>>>> 
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Ah, I see. The fix for that has been merged into master
so it will
>>>> be
>>>>>>>>> release in Flink 1.3.
>>>>>>>>> 
>>>>>>>>>> On 28. Apr 2017, at 13:50, Flavio Pompermaier <
>>>> pompermaier@okkam.it>
>>>>>>>>> wrote:
>>>>>>>>>> Sorry, you're right Aljosha..the issue number is
correct, the link
>>>> is
>>>>>>>>>> wrong! The correct one is https://issues.apache.org/
>>>>>>>>> jira/browse/FLINK-6398
>>>>>>>>>> On Fri, Apr 28, 2017 at 11:48 AM, Aljoscha Krettek
<
>>>>>>>> aljoscha@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> I think there might be a typo. We haven’t yet
reached issue
>> number
>>>>>>>> 6389,
>>>>>>>>>>> if I’m not mistaken. The latest as I’m writing
this is 6410.
>>>>>>>>>>> 
>>>>>>>>>>>> On 28. Apr 2017, at 10:00, Flavio Pompermaier
<
>>>>>>> pompermaier@okkam.it>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> If it's not a problem it will be great for
us to include also
>>>>>>>>> FLINK-6398
>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6938>
if it's not
>> a
>>>>>>> big
>>>>>>>>>>> deal
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Flavio
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Apr 28, 2017 at 3:32 AM, Zhuoluo
Yang <
>>>>>>>>>>> zhuoluo.yzl@alibaba-inc.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Devs,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for the release plan.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Could you also please add the feature
FLINK-6196
>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6196>
Support
>>>>>>> dynamic
>>>>>>>>>>> schema
>>>>>>>>>>>>> in Table Function?
>>>>>>>>>>>>> I’d like to update the code as comments
left on PR today.
>>>>>>>>>>>>> I will try to make sure the code is updated
before the Apr
>> 30th.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Zhuoluo 😀
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 在 2017年4月28日,上午8:48,Haohui
Mai <ricetons@gmail.com> 写道:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for starting this thread. It would
be great to see the
>>>>>>>>> following
>>>>>>>>>>>>> features available in Flink 1.3:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> * Support for complex schema: FLINK-6033,
FLINK-6377
>>>>>>>>>>>>> * Various improvements on SQL over group
windows: FLINK-6335,
>>>>>>>>> FLINK-6373
>>>>>>>>>>>>> * StreamTableSink for JDBC and Cassandra:
FLINK-6281,
>> FLINK-6225
>>>>>>>>>>>>> * Decoupling Flink and Hadoop: FLINK-5998
>>>>>>>>>>>>> 
>>>>>>>>>>>>> All of them have gone through at least
one round of review so
>>>> I'm
>>>>>>>>>>>>> optimistic that they can make it to 1.3
in a day or two.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Additionally it would be great to see
FLINK-6232 go in, but it
>>>>>>>> depends
>>>>>>>>>>> on
>>>>>>>>>>>>> FLINK-5884 so it might be a little bit
tough.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Haohui
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Apr 27, 2017 at 12:22 PM Chesnay
Schepler <
>>>>>>>> chesnay@apache.org
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> FLINK-5892 (Restoring state by operator)
is also nearing
>>>>>>> completion,
>>>>>>>>> but
>>>>>>>>>>>>> with only 1 day left before the weekend
we're cutting it really
>>>>>>>> short.
>>>>>>>>>>>>> Since this eliminates a major pain point
when updating jobs, as
>>>> it
>>>>>>>>>>>>> allows the modification of chains, another
day or 2 would be
>>>> good
>>>>>>> i
>>>>>>>>>>> think.
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Chesnay
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 27.04.2017 18:55, Bowen Li wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Ufuk,
>>>>>>>>>>>>> I'd like to get FLINK-6013 (Adding Datadog
Http metrics
>>>>>>> reporter)
>>>>>>>>>>>>> into
>>>>>>>>>>>>> 
>>>>>>>>>>>>> release 1.3. It's in the final state
of code review in
>>>>>>>>>>>>> https://github.com/apache/flink/pull/3736
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Bowen
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Thu, Apr 27, 2017 at 8:38 AM, Zhijiang(wangzhijiang999)
<
>>>>>>>>>>>>> wangzhijiang999@aliyun.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi Ufuk,
>>>>>>>>>>>>> Thank you for launching this topic!
>>>>>>>>>>>>> I wish my latest refinement of buffer
provider (
>>>>>>>>>>>>> 
>>>>>>>>>>>>> https://issues.apache.org/
>>>>>>>>>>>>> 
>>>>>>>>>>>>> jira/browse/FLINK-6337) to be included
in 1.3 and most of the
>>>>>>> jobs
>>>>>>>>> can
>>>>>>>>>>>>> get benefit from it. And I think it can
be completed with the
>>>> help
>>>>>>>> of
>>>>>>>>>>>>> your
>>>>>>>>>>>>> 
>>>>>>>>>>>>> reviews this week.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Cheers,Zhijiang---------------------------------------------
>>>>>>>>>>>>> ---------------------发件人:Ufuk
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Celebi <uce@apache.org>发送时间:2017年4月27日(星期四)
22:25收件人:dev <
>>>>>>>>>>>>> dev@flink.apache.org>抄 送:Robert
Metzger <rmetzger@apache.org>主
>>>>>>>>>>>>> 题:[DISCUSS] Feature Freeze
>>>>>>>>>>>>> Hey devs! :-)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> We decided to follow a time-based release
model with the
>>>> upcoming
>>>>>>>> 1.3
>>>>>>>>>>>>> release and the planned feature freeze
is on Monday, May 1st.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I wanted to start a discussion to get
a quick overview of the
>>>>>>>> current
>>>>>>>>>>>>> state of things.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> - Is everyone on track and aware of the
feature freeze? ;)
>>>>>>>>>>>>> - Are there any major features we want
in 1.3 that
>>>>>>>>>>>>> have not been merged yet?
>>>>>>>>>>>>> - Do we need to extend the feature freeze,
because of an
>>>>>>>>>>>>> important feature?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Would be great to gather a list of features/PRs
that we want in
>>>>>>> the
>>>>>>>>>>>>> 1.3 release. This could be a good starting
point for the
>> release
>>>>>>>>>>>>> manager (@Robert?).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Best,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Ufuk
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message