spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yafeng Guo <daniel.yafeng....@gmail.com>
Subject Re: Intellij IDEA 14 env setup; NoClassDefFoundError when run examples
Date Sun, 01 Feb 2015 13:29:26 GMT
Finally it works.

@Sean, I'm trying to setup env in IDE so I can track into to Spark -- that
will help me understand Spark internal mechanism.

@Ted, thanks. I'm using Maven, not SBT, but thanks for the suggestion
anyway.

For others who might interested in:

I choose "bigtop-dist" profile so under "Spark-Assembly" maven will build a
fat jar file, which include all artifacts built. Then I added such fat jar
as a dependency of examples module, set it as "runtime".

Then I find I can debug and track code step by step.

This is maybe not a formal way but at least works for me.

Regards,
Ya-Feng

On Sun, Feb 1, 2015 at 6:44 PM, Sean Owen <sowen@cloudera.com> wrote:

> How do you mean you run LogQuery? you would run these using the
> run-example script rather than in IntelliJ.
>
> On Sun, Feb 1, 2015 at 4:01 AM, Yafeng Guo <daniel.yafeng.guo@gmail.com>
> wrote:
> > Hi,
> >
> > I'm setting up a dev environment with Intellij IDEA 14. I selected
> profile
> > scala-2.10, maven-3, hadoop 2.4, hive, hive 0.13.1. The compilation
> passed.
> > But when I try to run LogQuery in examples, I met below issue:
> >
> > Connected to the target VM, address: '127.0.0.1:37182', transport:
> 'socket'
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > org/apache/spark/SparkConf
> > at org.apache.spark.examples.LogQuery$.main(LogQuery.scala:46)
> > at org.apache.spark.examples.LogQuery.main(LogQuery.scala)
> > Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > ... 2 more
> > Disconnected from the target VM, address: '127.0.0.1:37182', transport:
> > 'socket'
> >
> > anyone met similar issue before? Thanks a lot
> >
> > Regards,
> > Ya-Feng
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message