Return-Path: X-Original-To: apmail-flink-dev-archive@www.apache.org Delivered-To: apmail-flink-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 77D881798D for ; Tue, 9 Jun 2015 23:33:00 +0000 (UTC) Received: (qmail 53043 invoked by uid 500); 9 Jun 2015 23:33:00 -0000 Delivered-To: apmail-flink-dev-archive@flink.apache.org Received: (qmail 52976 invoked by uid 500); 9 Jun 2015 23:33:00 -0000 Mailing-List: contact dev-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list dev@flink.apache.org Received: (qmail 52965 invoked by uid 99); 9 Jun 2015 23:33:00 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Jun 2015 23:33:00 +0000 Received: from [10.0.1.73] (ipservice-092-214-156-009.092.214.pools.vodafone-ip.de [92.214.156.9]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 6900E1A0040 for ; Tue, 9 Jun 2015 23:32:58 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Testing Apache Flink 0.9.0-rc1 From: Ufuk Celebi In-Reply-To: <86B96B16-4CBD-4F5F-A346-B74D5557DA46@icloud.com> Date: Wed, 10 Jun 2015 01:32:56 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5B425CB2-544B-481C-8437-8BABB03F4A8B@icloud.com> <863324CF-051C-4DB3-BDF9-4BC4EFCEB759@apache.org> <86B96B16-4CBD-4F5F-A346-B74D5557DA46@icloud.com> To: dev@flink.apache.org X-Mailer: Apple Mail (2.1878.6) While looking into FLINK-2188 (HBase input) I've discovered that Hadoop = input formats implementing Configurable (like = mapreduce.TableInputFormat) don't have the Hadoop configuration set via = setConf(Configuration). I have a small fix for this, which I have to clean up. First, I wanted = to check what you think about this issue wrt the release. Personally, I = think this is a release blocker, because it essentially means that no = Hadoop input format, which relies on the Configuration instance to be = set this way will work (this is to some extent a bug of the respective = input formats) =96 most notably the HBase TableInputFormat. =96 Ufuk On 09 Jun 2015, at 18:07, Chiwan Park wrote: > I attached jps and jstack log about hanging = TaskManagerFailsWithSlotSharingITCase to JIRA FLINK-2183. >=20 > Regards, > Chiwan Park >=20 >> On Jun 10, 2015, at 12:28 AM, Aljoscha Krettek = wrote: >>=20 >> I discovered something that might be a feature, rather than a bug. = When you >> submit an example using the web client without giving parameters the >> program fails with this: >>=20 >> org.apache.flink.client.program.ProgramInvocationException: The main = method >> caused an error. >>=20 >> at >> = org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedPro= gram.java:452) >>=20 >> at >> = org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForEx= ecution(PackagedProgram.java:353) >>=20 >> at org.apache.flink.client.program.Client.run(Client.java:315) >>=20 >> at >> = org.apache.flink.client.web.JobSubmissionServlet.doGet(JobSubmissionServle= t.java:302) >>=20 >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) >>=20 >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:770) >>=20 >> at = org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:532) >>=20 >> at >> = org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)= >>=20 >> at >> = org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.ja= va:227) >>=20 >> at >> = org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.ja= va:965) >>=20 >> at = org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:388) >>=20 >> at >> = org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.jav= a:187) >>=20 >> at >> = org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.jav= a:901) >>=20 >> at >> = org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:1= 17) >>=20 >> at = org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47) >>=20 >> at >> = org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java= :113) >>=20 >> at org.eclipse.jetty.server.Server.handle(Server.java:352) >>=20 >> at >> = org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:= 596) >>=20 >> at >> = org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(Http= Connection.java:1048) >>=20 >> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:549) >>=20 >> at = org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:211) >>=20 >> at = org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:425) >>=20 >> at >> = org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.j= ava:489) >>=20 >> at >> = org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java= :436) >>=20 >> at java.lang.Thread.run(Thread.java:745) >>=20 >> Caused by: java.lang.NullPointerException >>=20 >> at >> = org.apache.flink.api.common.JobExecutionResult.getAccumulatorResult(JobExe= cutionResult.java:78) >>=20 >> at org.apache.flink.api.java.DataSet.collect(DataSet.java:409) >>=20 >> at org.apache.flink.api.java.DataSet.print(DataSet.java:1345) >>=20 >> at >> = org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:80)= >>=20 >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>=20 >> at >> = sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:= 62) >>=20 >> at >> = sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm= pl.java:43) >>=20 >> at java.lang.reflect.Method.invoke(Method.java:497) >>=20 >> at >> = org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedPro= gram.java:437) >>=20 >> ... 24 more >>=20 >>=20 >> This also only occurs when you uncheck the "suspend execution while = showing >> plan". >>=20 >> I think this arises because the new print() uses collect() which = tries to >> get the job execution result. I guess the result is Null since the = job is >> submitted asynchronously when the checkbox is unchecked. >>=20 >>=20 >> Other than that, the new print() is pretty sweet when you run the = builtin >> examples from the CLI. You get all the state changes and also the = result, >> even when running in cluster mode on several task managers. :D >>=20 >>=20 >> On Tue, Jun 9, 2015 at 3:41 PM, Aljoscha Krettek = >> wrote: >>=20 >>> I discovered another problem: >>> https://issues.apache.org/jira/browse/FLINK-2191 The closure cleaner >>> cannot be disabled in part of the Streaming Java API and all of the >>> Streaming Scala API. I think this is a release blocker (in addition >>> the the other bugs found so far.) >>>=20 >>> On Tue, Jun 9, 2015 at 2:35 PM, Aljoscha Krettek = >>> wrote: >>>> I found the bug in the failing YARNSessionFIFOITCase: It was = comparing >>>> the hostname to a hostname in some yarn config. In one case it was >>>> capitalised, in the other case it wasn't. >>>>=20 >>>> Pushing fix to master and release-0.9 branch. >>>>=20 >>>> On Tue, Jun 9, 2015 at 2:18 PM, Sachin Goel = >>> wrote: >>>>> A re-ran lead to reproducibility of 11 failures again. >>>>> TaskManagerTest.testSubmitAndExecuteTask was failing with a = time-out but >>>>> managed to succeed in a re-run. Here is the log output again: >>>>> http://pastebin.com/raw.php?i=3DN4cm1J18 >>>>>=20 >>>>> Setup: JDK 1.8.0_40 on windows 8.1 >>>>> System memory: 8GB, quad-core with maximum 8 threads. >>>>>=20 >>>>> Regards >>>>> Sachin Goel >>>>>=20 >>>>> On Tue, Jun 9, 2015 at 5:34 PM, Ufuk Celebi = wrote: >>>>>=20 >>>>>>=20 >>>>>> On 09 Jun 2015, at 13:58, Sachin Goel >>> wrote: >>>>>>=20 >>>>>>> On my local machine, several flink runtime tests are failing on = "mvn >>>>>> clean >>>>>>> verify". Here is the log output: >>> http://pastebin.com/raw.php?i=3DVWbx2ppf >>>>>>=20 >>>>>> Thanks for reporting this. Have you tried it multiple times? Is = it >>> failing >>>>>> reproducibly with the same tests? What's your setup? >>>>>>=20 >>>>>> =96 Ufuk >>>=20 >=20 >=20 >=20