hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: NoClassDefFoundError org/apache/hadoop/hbase/HBaseConfiguration
Date Wed, 06 Jul 2016 02:17:46 GMT

When using `spark-submit`, the application jar along with any jars included
with the `--jars` option
will be automatically transferred to the cluster. URLs supplied after
`--jars` must be separated by commas. That list is included on the driver
and executor classpaths.     Directory expansion does not work with

For spark.driver.extraClassPath :

Extra classpath entries to prepend to the classpath of the driver.
*Note:* In client mode, this config must not be set through the
SparkConf directly
in your application, because the driver JVM has already started at that
point. Instead, please set this through the --driver-class-path command
line option or in your default properties file.


On Tue, Jul 5, 2016 at 3:39 PM, Robert James <srobertjames@gmail.com> wrote:

> I'm using spark-shell.  The perplexing thing is that if I load it via
> spark-shell --jars, it seems to work.  However, if I load it via
> spark.driver.extraClassPath in the config file, it seems to fail.
> What is the difference between --jars (command line) and
> spark.driver.extraClassPath (config)?
> On 7/5/16, Dima Spivak <dspivak@cloudera.com> wrote:
> > Hey Robert,
> >
> > HBaseConfiguration is part of the hbase-common module of the HBase
> project.
> > Are you using Maven to provide dependencies or just running java -cp?
> >
> > -Dima
> >
> > On Monday, July 4, 2016, Robert James <srobertjames@gmail.com> wrote:
> >
> >> When trying to load HBase via Spark, I get NoClassDefFoundError
> >> org/apache/hadoop/hbase/HBaseConfiguration errors.
> >>
> >> How do I provide that class to Spark?
> >>
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message