flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Günter Hipler <guenter.hip...@unibas.ch>
Subject Re: problems starting the training exercise TaxiRideCleansing on local cluster
Date Sun, 09 Jul 2017 15:32:25 GMT
Thanks for response.

My classpath contains a version

  mvn dependency:build-classpath
[INFO] Scanning for projects...
[INFO]
[INFO] 
------------------------------------------------------------------------
[INFO] Building Flink Quickstart Job 0.1
[INFO] 
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:build-classpath (default-cli) @ 
flink-java-project ---
[INFO] Dependencies classpath:
<snip>

togram/2.1.6/HdrHistogram-2.1.6.jar:/home/swissbib/.m2/repository/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.jar:/home/swissbib/.m2/repository/joda-time/joda-time/2.9.9/joda-time-2.9.9.jar:

<snip>

which contains definitely the required method. 
(http://www.joda.org/joda-time/apidocs/org/joda/time/format/DateTimeFormatter.html#withZoneUTC--)

Something else is going wrong. I guess the way how I started (or 
configured) the local cluster (but it's done as described in the 
training setup (http://training.data-artisans.com/devEnvSetup.html) - 
which is very straightforward.

Günter


On 09.07.2017 16:17, Ted Yu wrote:
> Since the exception was about a missing method (withZoneUTC) instead 
> of class not found, it was likely due to a conflicting joda time jar 
> being on the classpath.
>
> Cheers
>
> On Sun, Jul 9, 2017 at 1:22 AM, Günter Hipler 
> <guenter.hipler@unibas.ch <mailto:guenter.hipler@unibas.ch>> wrote:
>
>     Hi,
>
>     sorry for this newbie question...
>
>     I'm following the data artisans exercises and wanted to run the
>     TaxiRide Cleansing job on my local cluster (version 1.3.1)
>     (http://training.data-artisans.com/exercises/rideCleansing.html
>     <http://training.data-artisans.com/exercises/rideCleansing.html>)
>
>     While this is possible within my IDE the cluster throws an
>     exception because of a missing type although the missed type is
>     part of the application jar the cluster is provided with.
>
>     swissbib@ub-sbhp02:~/environment/code/flink_einarbeitung/training/flink-java-project/target$
>     jar tf flink-java-project-0.1.jar | grep DateTimeFormatter
>     org/elasticsearch/common/joda/FormatDateTimeFormatter.class
>     org/joda/time/format/DateTimeFormatter.class
>     org/joda/time/format/DateTimeFormatterBuilder$CharacterLiteral.class
>     org/joda/time/format/DateTimeFormatterBuilder$Composite.class
>     org/joda/time/format/DateTimeFormatterBuilder$FixedNumber.class
>     org/joda/time/format/DateTimeFormatterBuilder$Fraction.class
>     org/joda/time/format/DateTimeFormatterBuilder$MatchingParser.class
>     org/joda/time/format/DateTimeFormatterBuilder$NumberFormatter.class
>     org/joda/time/format/DateTimeFormatterBuilder$PaddedNumber.class
>     org/joda/time/format/DateTimeFormatterBuilder$StringLiteral.class
>     org/joda/time/format/DateTimeFormatterBuilder$TextField.class
>     org/joda/time/format/DateTimeFormatterBuilder$TimeZoneId.class
>     org/joda/time/format/DateTimeFormatterBuilder$TimeZoneName.class
>     org/joda/time/format/DateTimeFormatterBuilder$TimeZoneOffset.class
>     org/joda/time/format/DateTimeFormatterBuilder$TwoDigitYear.class
>     org/joda/time/format/DateTimeFormatterBuilder$UnpaddedNumber.class
>     org/joda/time/format/DateTimeFormatterBuilder.class
>
>     Any advice? Thanks!
>
>     Günter
>
>
>     swissbib@ub-sbhp02:/usr/local/swissbib/flink$ bin/flink run -c
>     org.apache.flink.quickstart.St
>     <http://org.apache.flink.quickstart.St>reamingJob
>     /home/swissbib/environment/code/flink_einarbeitung/training/flink-java-project/target/flink-java-project-0.1.jar
>     --input
>     /home/swissbib/environment/code/flink_einarbeitung/training/flink-java-project/data/nycTaxiRides.gz
>
>     SLF4J: Class path contains multiple SLF4J bindings.
>     SLF4J: Found binding in
>     [jar:file:/usr/local/swissbib/flink-1.3.1/lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>     SLF4J: Found binding in
>     [jar:file:/home/swissbib/environment/tools/hbase-1.2.1/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>     SLF4J: Found binding in
>     [jar:file:/home/swissbib/environment/tools/hadoop-2.5.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>     SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
>     <http://www.slf4j.org/codes.html#multiple_bindings> for an
>     explanation.
>     SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>     Cluster configuration: Standalone cluster with JobManager at
>     localhost/127.0.0.1:6123 <http://127.0.0.1:6123>
>     Using address localhost:6123 to connect to JobManager.
>     JobManager web interface address http://localhost:8081
>     Starting execution of program
>     Submitting job with JobID: 32c7f2d0bbcac4d8c0367639ea928014.
>     Waiting for job completion.
>     Connected to JobManager at
>     Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-1464375722]
>     with leader session id 00000000-0000-0000-0000-000000000000.
>     07/09/2017 09:31:51    Job execution switched to status RUNNING.
>     07/09/2017 09:31:51    Source: Custom Source -> Filter -> Sink:
>     Unnamed(1/1) switched to SCHEDULED
>     07/09/2017 09:31:51    Source: Custom Source -> Filter -> Sink:
>     Unnamed(1/1) switched to DEPLOYING
>     07/09/2017 09:31:51    Source: Custom Source -> Filter -> Sink:
>     Unnamed(1/1) switched to RUNNING
>     07/09/2017 09:31:51    Source: Custom Source -> Filter -> Sink:
>     Unnamed(1/1) switched to FAILED
>     java.lang.NoSuchMethodError:
>     org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.datatypes.TaxiRide.<clinit>(TaxiRide.java:43)
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.sources.TaxiRideSource.generateUnorderedStream(TaxiRideSource.java:142)
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.sources.TaxiRideSource.run(TaxiRideSource.java:113)
>         at
>     org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87)
>         at
>     org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:55)
>         at
>     org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:95)
>         at
>     org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:263)
>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
>         at java.lang.Thread.run(Thread.java:745)
>
>     07/09/2017 09:31:51    Job execution switched to status FAILING.
>     java.lang.NoSuchMethodError:
>     org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.datatypes.TaxiRide.<clinit>(TaxiRide.java:43)
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.sources.TaxiRideSource.generateUnorderedStream(TaxiRideSource.java:142)
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.sources.TaxiRideSource.run(TaxiRideSource.java:113)
>         at
>     org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87)
>         at
>     org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:55)
>         at
>     org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:95)
>         at
>     org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:263)
>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
>         at java.lang.Thread.run(Thread.java:745)
>     07/09/2017 09:31:51    Job execution switched to status FAILED.
>
>     ------------------------------------------------------------
>      The program finished with the following exception:
>
>     org.apache.flink.client.program.ProgramInvocationException: The
>     program execution failed: Job execution failed.
>         at
>     org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:478)
>         at
>     org.apache.flink.client.program.StandaloneClusterClient.submitJob(StandaloneClusterClient.java:105)
>         at
>     org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:442)
>         at
>     org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:73)
>         at org.apache.flink.quickstart.St
>     <http://org.apache.flink.quickstart.St>reamingJob.main(StreamingJob.java:81)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
>     org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
>         at
>     org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:419)
>         at
>     org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:381)
>         at
>     org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:838)
>         at org.apache.flink.client.CliFrontend.run(CliFrontend.java:259)
>         at
>     org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1086)
>         at
>     org.apache.flink.client.CliFrontend$2.call(CliFrontend.java:1133)
>         at
>     org.apache.flink.client.CliFrontend$2.call(CliFrontend.java:1130)
>         at
>     org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.do
>     <http://javax.security.auth.Subject.do>As(Subject.java:422)
>         at
>     org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
>         at
>     org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
>         at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1130)
>     Caused by: org.apache.flink.runtime.client.JobExecutionException:
>     Job execution failed.
>         at
>     org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply$mcV$sp(JobManager.scala:933)
>         at
>     org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:876)
>         at
>     org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:876)
>         at
>     scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
>         at
>     scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
>         at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
>         at
>     akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
>         at
>     scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>         at
>     scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>         at
>     scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>         at
>     scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>     Caused by: java.lang.NoSuchMethodError:
>     org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.datatypes.TaxiRide.<clinit>(TaxiRide.java:43)
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.sources.TaxiRideSource.generateUnorderedStream(TaxiRideSource.java:142)
>         at
>     com.dataartisans.flinktraining.exercises.datastream_java.sources.TaxiRideSource.run(TaxiRideSource.java:113)
>         at
>     org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87)
>         at
>     org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:55)
>         at
>     org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:95)
>         at
>     org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:263)
>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>     -- 
>     Universität Basel
>     Universitätsbibliothek
>     Günter Hipler
>     Projekt SwissBib
>     Schoenbeinstrasse 18-20
>     4056 Basel, Schweiz
>     Tel.: + 41 (0)61 267 31 12
>     <tel:%2B%2041%20%280%2961%20267%2031%2012> Fax: ++41 61 267 3103
>     <tel:%2B%2B41%2061%20267%203103>
>     E-Mail guenter.hipler@unibas.ch <mailto:guenter.hipler@unibas.ch>
>     URL: www.swissbib.org <http://www.swissbib.org> /
>     http://www.ub.unibas.ch/
>
>

-- 
Universität Basel
Universitätsbibliothek
Günter Hipler
Projekt SwissBib
Schoenbeinstrasse 18-20
4056 Basel, Schweiz
Tel.: + 41 (0)61 267 31 12 Fax: ++41 61 267 3103
E-Mail guenter.hipler@unibas.ch
URL: www.swissbib.org  / http://www.ub.unibas.ch/


Mime
View raw message