flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Cluster installation gives java.lang.NoClassDefFoundError for everything
Date Wed, 11 Nov 2015 12:16:22 GMT
Is "jar tf /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar"
listing for example the "org/apache/flink/client/CliFrontend" class?

On Wed, Nov 11, 2015 at 12:09 PM, Maximilian Michels <mxm@apache.org> wrote:

> Hi Camelia,
>
> Flink 0.9.X supports Java 6. So this can't be the issue.
>
> Out of curiosity, I gave it a spin on a Linux machine with OpenJDK 6. I
> was able to start the command-line interface, job manager and task managers.
>
> java version "1.6.0_36"
> OpenJDK Runtime Environment (IcedTea6 1.13.8) (6b36-1.13.8-0ubuntu1~14.04)
> OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
>
>
> I think the error must be caused by your NFS setup. Let's start with
> ./bin/flink. For instance, can you access
> "/users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar" from the
> machine where you run ./bin/flink?
>
> Best,
> Max
>
>
> On Wed, Nov 11, 2015 at 10:41 AM, Camelia Elena Ciolac <
> camelia@chalmers.se> wrote:
>
>>  Hello,
>>
>> As promised, I come back with debugging details.
>> So:
>>
>> ******* In start-cluster.sh , the following echo's
>>
>> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
>> echo $HOSTLIST
>> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
>> echo $FLINK_BIN_DIR
>> echo "start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
>>
>> # cluster mode, bring up job manager locally and a task manager on every
>> slave host
>> "$FLINK_BIN_DIR"/jobmanager.sh start cluster batch
>>
>> ==>  gave:
>>
>> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> /users/camelia/thecluster/flink-0.9.1/conf/slaves
>> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> /users/camelia/thecluster/flink-0.9.1/bin
>> start-cluster ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>
>> ******* In jobmanager.sh , the following echo's
>>
>> echo "Starting Job Manager"
>>         echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
>>         pwd
>>         echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
>>         echo ${FLINK_ENV_JAVA_OPTS}
>>         echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
>>         echo $FLINK_JM_CLASSPATH
>>         echo "jobmanager ~~~~~~~~~~~~~~~~~~~~~"
>>         echo $INTERNAL_HADOOP_CLASSPATHS
>>         echo "jobmanager ~~~~~~~~~~~~~~~~~~~~"
>>         echo $FLINK_CONF_DIR
>>         echo "~~~~~~~~~~~~~~~~~~~~"
>>         $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}"
>> -classpath "`manglePathList
>> "$FLINK_JM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
>> org.apache.flink.runtime.jobmanager.JobManager --configDir
>> "$FLINK_CONF_DIR" --executionMode $EXECUTIONMODE --streamingMode
>> "$STREAMINGMODE" > "$out" 2>&1 < /dev/null &
>>
>>
>>
>> ==> gave:
>>
>> jobmanager ~~~~~~~~~~~~~~~~~~~~~
>> /users/camelia/thecluster
>> jobmanager ~~~~~~~~~~~~~~~~~~~~~
>>
>> jobmanager ~~~~~~~~~~~~~~~~~~~~~
>>
>> /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
>> jobmanager ~~~~~~~~~~~~~~~~~~~~~
>> ::
>> jobmanager ~~~~~~~~~~~~~~~~~~~~
>> /users/camelia/thecluster/flink-0.9.1/conf
>> ~~~~~~~~~~~~~~~~~~~~
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/flink/runtime/jobmanager/JobManager
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flink.runtime.jobmanager.JobManager
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
>> Could not find the main class:
>> org.apache.flink.runtime.jobmanager.JobManager. Program will exit.
>>
>>
>>
>> ******* In taskmanager.sh, the following echo's
>>
>> echo Starting task manager on host $HOSTNAME
>>         echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
>>         pwd
>>         echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
>>         echo ${FLINK_ENV_JAVA_OPTS}
>>         echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
>>         echo $FLINK_JM_CLASSPATH
>>         echo "taskmanager ~~~~~~~~~~~~~~~~~~~~~"
>>         echo $INTERNAL_HADOOP_CLASSPATHS
>>         echo "taskmanager ~~~~~~~~~~~~~~~~~~~~"
>>         echo $FLINK_CONF_DIR
>>         echo "~~~~~~~~~~~~~~~~~~~~"
>>         $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}"
>> -classpath "`manglePathList
>> "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
>> org.apache.flink.runtime.taskmanager.TaskManager --configDir
>> "$FLINK_CONF_DIR" --streamingMode "$STREAMINGMODE" > "$out" 2>&1 <
>> /dev/null &
>>
>> ==> gave:
>>
>> taskmanager ~~~~~~~~~~~~~~~~~~~~~
>> /users/camelia/thecluster
>> taskmanager ~~~~~~~~~~~~~~~~~~~~~
>>
>> taskmanager ~~~~~~~~~~~~~~~~~~~~~
>>
>> taskmanager ~~~~~~~~~~~~~~~~~~~~~
>> ::
>> taskmanager ~~~~~~~~~~~~~~~~~~~~
>> /users/camelia/thecluster/flink-0.9.1/conf
>> ~~~~~~~~~~~~~~~~~~~~
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/flink/runtime/taskmanager/TaskManager
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flink.runtime.taskmanager.TaskManager
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
>> Could not find the main class:
>> org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.
>>
>>
>>
>> ******* In flink , the following echo's
>>
>> echo "flink ~~~~~~~~~~~~~~~~~~~~~"
>> pwd
>> echo "flink ~~~~~~~~~~~~~~~~~~~~~"
>> echo $CC_CLASSPATH
>> echo "flink ~~~~~~~~~~~~~~~~~~~~~"
>> echo $INTERNAL_HADOOP_CLASSPATHS
>> echo "~~~~~~~~~~~~~~~~~~~~"
>> $JAVA_RUN $JVM_ARGS "${log_setting[@]}" -classpath "`manglePathList
>> "$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
>> org.apache.flink.client.CliFrontend "$@"
>>
>>
>> ==> gave:
>>
>> flink ~~~~~~~~~~~~~~~~~~~~~
>> /users/camelia/thecluster
>> flink ~~~~~~~~~~~~~~~~~~~~~
>>
>> /users/camelia/thecluster/flink-0.9.1/lib/flink-dist-0.9.1.jar:/users/camelia/thecluster/flink-0.9.1/lib/flink-python-0.9.1.jar
>> flink ~~~~~~~~~~~~~~~~~~~~~
>> ::
>> ~~~~~~~~~~~~~~~~~~~~
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/flink/client/CliFrontend
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flink.client.CliFrontend
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
>> Could not find the main class: org.apache.flink.client.CliFrontend.
>> Program will exit.
>>
>>
>> I hope that now you have a better image of the issue and I hope you can
>> provide some guidance to solve it.
>> I mention that I use the simplest scenario possible, just 2 nodes (1
>> master and 1 slave) without Hadoop for this test.
>> Also, I have a hostname command inside the script that shows me that I'm
>> on the master node when running these scripts.
>>
>> Thank you very much!
>> Camelia
>>
>>
>> ------------------------------
>> *From:* Camelia Elena Ciolac
>> *Sent:* Wednesday, November 11, 2015 9:01 AM
>> *To:* user@flink.apache.org
>> *Subject:* RE: Cluster installation gives java.lang.NoClassDefFoundError
>> for everything
>>
>> Good morning,
>>
>> Thank you Stephan!
>>
>> I keep on testing and in the meantime I'm wondering if the Java version
>> on the cluster may be part of the issue:
>> java version "1.6.0_36"
>> OpenJDK Runtime Environment (IcedTea6 1.13.8) (rhel-1.13.8.1.el6_7-x86_64)
>> OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
>> ?
>>
>> The cluster has shared network file system, so no problem in this aspect.
>> I keep on trying and I've put some echo's for debugging in the Flink
>> start scripts.
>>
>> Best regards,
>> Camelia
>>
>>
>>
>> ------------------------------
>> *From:* Camelia Elena Ciolac
>> *Sent:* Monday, November 09, 2015 2:32 PM
>> *To:* user@flink.apache.org
>> *Subject:* Cluster installation gives java.lang.NoClassDefFoundError for
>> everything
>>
>> Hello,
>>
>> I am configuring Flink to run on a cluster with NFS.
>> I have the Flink 0.9.1 distribution in some path in NFS and I added that
>> path in ~/.bashrc as FLINK_HOME, and also included the $FLINK_HOME/lib
>> folder to $PATH.
>> I have the slaves file and the yaml file configured correctly with the
>> nodes involved.
>>
>> Still, when I run the following command I get the following errors:
>>
>> $FLINK_HOME/bin/start-cluster.sh
>>
>> (tried also
>> cd $FLINK_HOME
>> ./bin/start-cluster.sh)
>>
>> --> in /log/flink-theuser-jobmanager-thenode.out
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/flink/runtime/jobmanager/JobManager
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flink.runtime.jobmanager.JobManager
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
>> Could not find the main class:
>> org.apache.flink.runtime.jobmanager.JobManager. Program will exit.
>>
>> -->in /log/flink-theuser-taskmanager-theslavenode.out
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/flink/runtime/taskmanager/TaskManager
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flink.runtime.taskmanager.TaskManager
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
>> Could not find the main class:
>> org.apache.flink.runtime.taskmanager.TaskManager. Program will exit.
>>
>>
>> And lastly, in my script I have the run of a job which gives:
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/flink/client/CliFrontend
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flink.client.CliFrontend
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
>> Could not find the main class: org.apache.flink.client.CliFrontend.
>> Program will exit.
>>
>> What could be the cause and which is the solution?
>>
>> Looking forward to your answer as soon as possible.
>> Many thanks,
>> Camelia
>>
>>
>

Mime
View raw message