flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: [DISCUSS] Release 1.3.0 RC0 (Non voting, testing release candidate)
Date Fri, 12 May 2017 07:52:03 GMT
Hi Haohui,

open issue for 1.3 are tracked in JIRA.
You can specify "Fix version: 1.3.0" and an appropriate priority.

I updated a couple of issues to have the fix version tag.
Any bug that is found while release testing should be added to this list by
setting fix version, priority, and component.

Thanks,
Fabian

PS.: This query will list the open Table API issues for 1.3:

https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%201.3.0%20AND%20component%20%3D%20%22Table%20API%20%26%20SQL%22%20ORDER%20BY%20priority%20DESC

2017-05-12 0:50 GMT+02:00 Haohui Mai <ricetons@gmail.com>:

> Hi,
>
> Just curious -- do we have anything to track the blockers on the table /
> SQL side? For example, it seems to me that FLINK-6563 is a blocker because
> all group windows in SQL will not work without it. I think that Fabian /
> Timo might want to get several jiras into 1.3.
>
> Regards,
> Haohui
>
> On Thu, May 11, 2017 at 12:33 PM Stephan Ewen <sewen@apache.org> wrote:
>
> > From the above list, I addressed a bunch of issues.
> >
> > - FLINK-6514 Cannot start Flink Cluster in standalone mode has an open
> Pull
> > Request:
> >    - master: https://github.com/apache/flink/pull/3876
> >    - 1.3: https://github.com/apache/flink/pull/3877
> >
> > Left are:
> >
> > - FLINK-6537 Umbrella issue for fixes to incremental snapshots (Stefan
> has
> > PR
> > open to fix the critical ones)
> >
> > - FLINK-6284 Incorrect sorting of completed checkpoints in
> > ZooKeeperCompletedCheckpointStore
> >
> >
> > *I think that FLINK-6284 is totally a blocker for the release!*
> >
> >
> > On Thu, May 11, 2017 at 4:19 PM, Robert Metzger <rmetzger@apache.org>
> > wrote:
> >
> > > It seems that we found quite a large number of critical issues in the
> > first
> > > RC.
> > >
> > > - FLINK-6537 Umbrella issue for fixes to incremental snapshots (Stefan
> > has
> > > a PR open to fix the critical ones)
> > > - FLINK-6531 Deserialize checkpoint hooks with user classloader (has a
> > > pending PR)
> > > - FLINK-6515 KafkaConsumer checkpointing fails because of ClassLoader
> > > issues (status unknown)
> > > - FLINK-6514 Cannot start Flink Cluster in standalone mode (Stephan is
> on
> > > it)
> > > - FLINK-6508 Include license files of packaged dependencies (Stephan is
> > on
> > > it) + FLINK-6501 Make sure NOTICE files are bundled into shaded JAR
> files
> > > - FLINK-6284 Incorrect sorting of completed checkpoints in
> > > ZooKeeperCompletedCheckpointStore (unknown)
> > >
> > > I would like to get these issues fixed by end of this week (Sunday), so
> > > that I can create the first voting RC on Monday morning.
> > > Please reject if you think we will not manage to get the stuff fixed
> > until
> > > then.
> > >
> > >
> > >
> > > On Thu, May 11, 2017 at 10:54 AM, Till Rohrmann <trohrmann@apache.org>
> > > wrote:
> > >
> > > > Unfortunately, it won't be fully functional in 1.3.
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Thu, May 11, 2017 at 10:45 AM, Renjie Liu <
> liurenjie2008@gmail.com>
> > > > wrote:
> > > >
> > > > > @Rohrmann Will FLIP 6 be fully functional in 1.3 release?
> > > > >
> > > > > On Thu, May 11, 2017 at 4:12 PM Gyula Fóra <gyula.fora@gmail.com>
> > > wrote:
> > > > >
> > > > > > Thanks Stefan!
> > > > > > Gyula
> > > > > >
> > > > > > Stefan Richter <s.richter@data-artisans.com> ezt írta
(időpont:
> > > 2017.
> > > > > máj.
> > > > > > 11., Cs, 10:04):
> > > > > >
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > Thanks for reporting this. I found a couple of issues yesterday
> > > and I
> > > > > am
> > > > > > > currently working on a bundle of fixes. I will take a look
at
> > this
> > > > > > problem,
> > > > > > > and if it is already covered.
> > > > > > >
> > > > > > > Best,
> > > > > > > Stefan
> > > > > > >
> > > > > > > > Am 11.05.2017 um 09:47 schrieb Gyula Fóra <
> > gyula.fora@gmail.com
> > > >:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > > I am not sure if this belong to this thread, but while
trying
> > to
> > > > run
> > > > > a
> > > > > > > job
> > > > > > > > with rocks incremental backend I ran into 2 issues:
> > > > > > > >
> > > > > > > > One with savepoints, I can't figure out because I
can't make
> > > sense
> > > > of
> > > > > > the
> > > > > > > > error or how it happenned:
> > > > > > > > The error stack trace is here:
> > > > > > > >
> > https://gist.github.com/gyfora/2f7bb387bbd9f455f9702908cde0b239
> > > > > > > > This happens on every savepoint attempt and seems
to be
> related
> > > to
> > > > > the
> > > > > > > > kafka source. Interestingly other tasks succeed in
writing
> data
> > > to
> > > > > > hdfs.
> > > > > > > >
> > > > > > > > The other one is covered by
> > > > > > > https://issues.apache.org/jira/browse/FLINK-6531 I
> > > > > > > > guess. I am not sure if the first one is related though.
> > > > > > > >
> > > > > > > > Thank you!
> > > > > > > > Gyula
> > > > > > > >
> > > > > > > > Till Rohrmann <trohrmann@apache.org> ezt írta
(időpont:
> 2017.
> > > máj.
> > > > > > 11.,
> > > > > > > Cs,
> > > > > > > > 9:14):
> > > > > > > >
> > > > > > > >> Hi Renjie,
> > > > > > > >>
> > > > > > > >> 1.3 already contains some Flip-6 code. However,
it is not
> yet
> > > > fully
> > > > > > > >> functional. What you already can do is to run
local jobs on
> > the
> > > > > Flip-6
> > > > > > > code
> > > > > > > >> base by instantiating a MiniCluster and then using
the
> > > > > > > >> Flip6LocalStreamEnvironment.
> > > > > > > >>
> > > > > > > >> Cheers,
> > > > > > > >> Till
> > > > > > > >>
> > > > > > > >> On Thu, May 11, 2017 at 6:00 AM, Renjie Liu <
> > > > > liurenjie2008@gmail.com>
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >>> Hi, all:
> > > > > > > >>> Will the FLIP 6 be included in release 1.3?
> > > > > > > >>>
> > > > > > > >>> On Wed, May 10, 2017 at 9:48 PM Gyula Fóra
<
> > > gyula.fora@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >>>
> > > > > > > >>>> Thanks you! :)
> > > > > > > >>>>
> > > > > > > >>>> Chesnay Schepler <chesnay@apache.org>
ezt írta (időpont:
> > > 2017.
> > > > > máj.
> > > > > > > >> 10.,
> > > > > > > >>>> Sze, 15:44):
> > > > > > > >>>>
> > > > > > > >>>>> I guess it is related to this one
> > > > > > > >>>>> https://issues.apache.org/jira/browse/FLINK-6514
?
> > > > > > > >>>>>
> > > > > > > >>>>> On 10.05.2017 15:34, Gyula Fóra wrote:
> > > > > > > >>>>>> Hi,
> > > > > > > >>>>>>
> > > > > > > >>>>>> I tried to run an application
on 1.3 but I keep getting
> > the
> > > > > > > >> following
> > > > > > > >>>>> error:
> > > > > > > >>>>>> java.lang.NoClassDefFoundError:
Could not initialize
> class
> > > > > > > >>>>>> org.apache.hadoop.security.UserGroupInformation
> > > > > > > >>>>>> at
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>> org.apache.flink.runtime.security.modules.HadoopModule.
> > > > > > > >>> install(HadoopModule.java:45)
> > > > > > > >>>>>> at
> > > > > > > >>>>>>
> > > > > > > >>>>>
> > > > > > > >>>> org.apache.flink.runtime.security.SecurityUtils.
> > > > > > > >>> install(SecurityUtils.java:78)
> > > > > > > >>>>>> at org.apache.flink.client.
> CliFrontend.main(CliFrontend.
> > > > > java:1128)
> > > > > > > >>>>>>
> > > > > > > >>>>>> Even after adding hadoop-common
to the lib manually
> > (which I
> > > > > guess
> > > > > > > >>>> should
> > > > > > > >>>>>> not be necessary).
> > > > > > > >>>>>>
> > > > > > > >>>>>> Any idea what might cause this?
> > > > > > > >>>>>>
> > > > > > > >>>>>> Thanks,
> > > > > > > >>>>>> Gyula
> > > > > > > >>>>>>
> > > > > > > >>>>>> Chesnay Schepler <chesnay@apache.org>
ezt írta
> (időpont:
> > > > 2017.
> > > > > > > >> máj.
> > > > > > > >>>> 9.,
> > > > > > > >>>>> K,
> > > > > > > >>>>>> 13:27):
> > > > > > > >>>>>>
> > > > > > > >>>>>>> I looked into AppVeyor and
managed to create a working
> > > setup
> > > > > that
> > > > > > > >>>>>>> replicates the problems i
encounter locally.
> > > > > > > >>>>>>>
> > https://ci.appveyor.com/project/zentol/flink/build/1.0.12
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> The integration should be
as simple as placing an
> > > > appveyor.yml
> > > > > > > >>>> (similar
> > > > > > > >>>>>>> to the travis.yml) into the
repository and asking INFRA
> > to
> > > > > > enabled
> > > > > > > >>> it
> > > > > > > >>>>>>> for Flink.
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> I've opened a JIRA to track
this initiative:
> > > > > > > >>>>>>> https://issues.apache.org/jira/browse/FLINK-6510
> > > > > > > >>>>>>>
> > > > > > > >>>>>>> On 09.05.2017 10:17, Robert
Metzger wrote:
> > > > > > > >>>>>>>> Chesnay, I would propose
to look into a CI solution
> for
> > > > > Windows.
> > > > > > > >>> I'll
> > > > > > > >>>>> try
> > > > > > > >>>>>>>> out appveyor.com for my
Flink fork and report back
> how
> > > well
> > > > > it
> > > > > > > >>> works
> > > > > > > >>>>>>> (let
> > > > > > > >>>>>>>> me know if you want to
look into it).
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>> On Tue, May 9, 2017 at
10:05 AM, Stephan Ewen <
> > > > > sewen@apache.org
> > > > > > >
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>>>>
> > > > > > > >>>>>>>>> @Renjie What functionality
are you looking for? Most
> > > stuff
> > > > > > > >> should
> > > > > > > >>>> work
> > > > > > > >>>>>>> in
> > > > > > > >>>>>>>>> the normal master
branch.
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>> On Tue, May 9, 2017
at 10:02 AM, Renjie Liu <
> > > > > > > >>>> liurenjie2008@gmail.com>
> > > > > > > >>>>>>>>> wrote:
> > > > > > > >>>>>>>>>
> > > > > > > >>>>>>>>>> Stephan:
> > > > > > > >>>>>>>>>> This means that
 we can use FLIP 6 for basic
> > > functionality
> > > > > > now?
> > > > > > > >>> Any
> > > > > > > >>>>>>>>>> document for setting
up that? Will FLIP 6 be
> finished
> > in
> > > > > > > >> release
> > > > > > > >>>> 1.3?
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>> On Tue, May 9,
2017 at 3:43 PM Stephan Ewen <
> > > > > sewen@apache.org
> > > > > > >
> > > > > > > >>>>> wrote:
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>>>>> @Renjie Liu:
A good part of the FLIP-6 code is in
> > > there,
> > > > > but
> > > > > > > >> not
> > > > > > > >>>>> all.
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> It does run
well on Yarn, Mesos, Docker, etc.
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> We need to
finish the FLIP-6 work mainly for full
> > > > > elasticity.
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>> On Tue, May
9, 2017 at 5:24 AM, Renjie Liu <
> > > > > > > >>>> liurenjie2008@gmail.com
> > > > > > > >>>>>>
> > > > > > > >>>>>>>>>>> wrote:
> > > > > > > >>>>>>>>>>>
> > > > > > > >>>>>>>>>>>> Hi, does
this include the FLIP6?
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>> On Tue,
May 9, 2017 at 2:29 AM Stephan Ewen <
> > > > > > > >> sewen@apache.org>
> > > > > > > >>>>>>> wrote:
> > > > > > > >>>>>>>>>>>>> Did
a quick test: Simply adding the
> > > > > > > >> "org.apache.maven.plugins.
> > > > > > > >>>>>>>>>>>>> shade.resource.ApacheNoticeResourceTransformer"
> > > helps
> > > > > with
> > > > > > > >>>> NOTICE
> > > > > > > >>>>>>>>>>>>> files,
> > > > > > > >>>>>>>>>>>>> but
does not add the required BSD licence copies.
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>> On
Mon, May 8, 2017 at 8:25 PM, Stephan Ewen <
> > > > > > > >>> sewen@apache.org>
> > > > > > > >>>>>>>>> wrote:
> > > > > > > >>>>>>>>>>>>>>
I did the first pass for the legal check.
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>
   - Source LICENSE and NOTICE are okay
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>
   - In the shaded JAR files, we are not
> bundling
> > > the
> > > > > > > >>> license
> > > > > > > >>>>> and
> > > > > > > >>>>>>>>>>>>>>
notice files of the dependencies we include in
> the
> > > > > shaded
> > > > > > > >>> jars.
> > > > > > > >>>>>>>>>>>>>>
      => Not a problem for Guava (Apache
> Licensed)
> > > > > > > >>>>>>>>>>>>>>
      => I think is a problem for ASM
> > > (redistribution
> > > > in
> > > > > > > >>>> binary
> > > > > > > >>>>>>> form,
> > > > > > > >>>>>>>>>>>>>>
hence needs a notice of the copy)
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>
   - The Table API / SQL module needs more
> entries
> > > for
> > > > > > > >>> Janino
> > > > > > > >>>> /
> > > > > > > >>>>>>>>>>>>>>
Reflections (both BSD licensed)
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>
So that is definitely a blocker.
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>
On Mon, May 8, 2017 at 12:14 PM, Robert Metzger
> <
> > > > > > > >>>>>>> rmetzger@apache.org
> > > > > > > >>>>>>>>>>>>>>
wrote:
> > > > > > > >>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
Hi Devs,
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
I've created a first non-voting release
> candidate
> > > for
> > > > > > > >> Flink
> > > > > > > >>>>> 1.3.0.
> > > > > > > >>>>>>>>>>>>>>>
Please use this RC to test as much as you can
> and
> > > > > provide
> > > > > > > >>>>> feedback
> > > > > > > >>>>>>>>> to
> > > > > > > >>>>>>>>>>>>>>>
the Flink community. The more we find and fix
> > now,
> > > > the
> > > > > > > >>> better
> > > > > > > >>>>>>> Flink
> > > > > > > >>>>>>>>> 1.3.0
> > > > > > > >>>>>>>>>>>>>>>
wil be :)
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
I've CC'ed the user@ mailing list to get more
> > > people
> > > > > to
> > > > > > > >>> test
> > > > > > > >>>>> it.
> > > > > > > >>>>>>> DO
> > > > > > > >>>>>>>>>>>>>>>
NOT USE THIS RELEASE CANDIDATE IN PRODUCTION.
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
I will prepare a google document to synchronize
> > the
> > > > > > > >> testing
> > > > > > > >>>>>>> effort a
> > > > > > > >>>>>>>>>>>>>>>
bit more.
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
Depending on the number of issues we identify,
> I
> > > hope
> > > > > > that
> > > > > > > >>> we
> > > > > > > >>>>> can
> > > > > > > >>>>>>> do
> > > > > > > >>>>>>>>>>>>>>>
the first VOTEing RC early next week.
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
------------------------------
> > > > > > > >>> ------------------------------
> > > > > > > >>>>>>>>> ---------
> > > > > > > >>>>>>>>>>>>>>>
The release commit is
> > > > > > > >>>> f94c002991dcce9f1104f8e61b43efb2f8247cb4,
> > > > > > > >>>>>>>>>>>>>>>
located here:
> > > > > > > >> http://git-wip-us.apache.org/repos/asf/flink/
> > > > > > > >>>>>>>>>>>>>>>
commit/f94c0029
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
The artifacts are located here:
> > > http://people.apache
> > > > .
> > > > > > > >>>>>>>>>>>>>>>
org/~rmetzger/flink-1.3.0-rc0/
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
The maven staging repository is here:
> > > > > https://repository
> > > > > > .
> > > > > > > >>>>>>>>>>>>>>>
apache.org/content/repositorie
> > > s/orgapacheflink-1118
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
------------------------------
> > > > > > > >>> ------------------------------
> > > > > > > >>>>>>>>> ---------
> > > > > > > >>>>>>>>>>>>>>>
Happy testing!
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
Regards,
> > > > > > > >>>>>>>>>>>>>>>
Robert
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>>>>
> > > > > > > >>>>>>>>>>>>> --
> > > > > > > >>>>>>>>>>>> Liu, Renjie
> > > > > > > >>>>>>>>>>>> Software
Engineer, MVAD
> > > > > > > >>>>>>>>>>>>
> > > > > > > >>>>>>>>>>> --
> > > > > > > >>>>>>>>>> Liu, Renjie
> > > > > > > >>>>>>>>>> Software Engineer,
MVAD
> > > > > > > >>>>>>>>>>
> > > > > > > >>>>>>>
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>> --
> > > > > > > >>> Liu, Renjie
> > > > > > > >>> Software Engineer, MVAD
> > > > > > > >>>
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > > --
> > > > > Liu, Renjie
> > > > > Software Engineer, MVAD
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message