spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng, Hao" <hao.ch...@intel.com>
Subject RE: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
Date Mon, 16 Mar 2015 07:22:46 GMT
It doesn’t take effect if just putting jar files under the lib-managed/jars folder, you need
to put that under class path explicitly.

From: sandeep vura [mailto:sandeepvura@gmail.com]
Sent: Monday, March 16, 2015 2:21 PM
To: Cheng, Hao
Cc: fightfate@163.com; Ted Yu; user
Subject: Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

I have already added mysql-connector-xx.jar file in spark/lib-managed/jars directory.

Regards,
Sandeep.v

On Mon, Mar 16, 2015 at 11:48 AM, Cheng, Hao <hao.cheng@intel.com<mailto:hao.cheng@intel.com>>
wrote:
Or you need to specify the jars either in configuration or

bin/spark-sql --jars  mysql-connector-xx.jar

From: fightfate@163.com<mailto:fightfate@163.com> [mailto:fightfate@163.com<mailto:fightfate@163.com>]
Sent: Monday, March 16, 2015 2:04 PM
To: sandeep vura; Ted Yu
Cc: user
Subject: Re: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Hi, Sandeep

From your error log I can see that jdbc driver not found in your classpath. Did you had your
mysql
jdbc jar correctly configured in the specific classpath? Can you establish a hive jdbc connection
using
the url : jdbc:hive2://localhost:10000 ?

Thanks,
Sun.

________________________________
fightfate@163.com<mailto:fightfate@163.com>

From: sandeep vura<mailto:sandeepvura@gmail.com>
Date: 2015-03-16 14:13
To: Ted Yu<mailto:yuzhihong@gmail.com>
CC: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
Hi Ted,

Did you find any solution.

Thanks
Sandeep

On Mon, Mar 16, 2015 at 10:44 AM, sandeep vura <sandeepvura@gmail.com<mailto:sandeepvura@gmail.com>>
wrote:
Hi Ted,

I am using Spark -1.2.1 and hive -0.13.1 you can check my configuration files attached below.

------------------------------------
ERROR IN SPARK
------------------------------------
n: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav         
                                                                              a:346)
        at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkS         
                                                                              QLCLIDriver.scala:101)
        at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQ         
                                                                              LCLIDriver.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.         
                                                                              java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces         
                                                                              sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:622)
        at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.h         
                                                                              ive.metastore.HiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore         
                                                                              Utils.java:1412)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(Retry   
                                                                                    ingMetaStoreClient.java:62)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(Ret         
                                                                              ryingMetaStoreClient.java:72)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.ja         
                                                                              va:2453)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav         
                                                                              a:340)
        ... 9 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct         
                                                                              orAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC         
                                                                              onstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:534)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStore         
                                                                              Utils.java:1410)
        ... 14 more
Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional con         
                                                                              nection factory
NestedThrowables:
java.lang.reflect.InvocationTargetException
        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusExc         
                                                                              eption(NucleusJDOHelper.java:587)
        at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfigurat         
                                                                              ion(JDOPersistenceManagerFactory.java:788)
        at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenc         
                                                                              eManagerFactory(JDOPersistenceManagerFactory.java:333)
        at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceMa         
                                                                              nagerFactory(JDOPersistenceManagerFactory.java:202)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.         
                                                                              java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces         
                                                                              sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:622)
        at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
        at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementatio         
                                                                              n(JDOHelper.java:1166)
        at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
        at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
        at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:         
                                                                              310)
        at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(Ob         
                                                                              jectStore.java:339)
        at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.j         
                                                                              ava:248)
        at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java         
                                                                              :223)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:6         
                                                                              2)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.ja         
                                                                              va:117)
        at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.j   
                                                                                    ava:58)
        at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy         
                                                                              .java:67)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore         
                                                                              (HiveMetaStore.java:497)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveM         
                                                                              etaStore.java:475)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefau         
                                                                              ltDB(HiveMetaStore.java:523)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMe         
                                                                              taStore.java:397)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(Hive   
                                                                                    MetaStore.java:356)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHM   
                                                                                    SHandler.java:54)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(Retrying         
                                                                              HMSHandler.java:59)
        at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMeta         
                                                                              Store.java:4944)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaS   
                                                                                    toreClient.java:171)
        ... 19 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct         
                                                                              orAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC         
                                                                              onstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:534)
        at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExten         
                                                                              sion(NonManagedPluginRegistry.java:631)
        at org.datanucleus.plugin.PluginManager.createExecutableExtension(Plugin         
                                                                              Manager.java:325)
        at org.datanucleus.store.AbstractStoreManager.registerConnectionFactory(         
                                                                              AbstractStoreManager.java:282)
        at org.datanucleus.store.AbstractStoreManager.<init>(AbstractStoreManage   
                                                                                    r.java:240)
        at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManage   
                                                                                    r.java:286)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct         
                                                                              orAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC         
                                                                              onstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:534)
        at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExten         
                                                                              sion(NonManagedPluginRegistry.java:631)
        at org.datanucleus.plugin.PluginManager.createExecutableExtension(Plugin         
                                                                              Manager.java:301)
        at org.datanucleus.NucleusContext.createStoreManagerForProperties(Nucleu         
                                                                              sContext.java:1187)
        at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
        at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfigurat         
                                                                              ion(JDOPersistenceManagerFactory.java:775)
        ... 48 more
Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "B         
                                                                              ONECP" plugin
to create a ConnectionPool gave an error : The specified datastore                       
                                                                 driver ("com.mysql.jdbc.Driver")
was not found in the CLASSPATH. Please check y                                           
                                            our CLASSPATH specification, and the name of the
driver.
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources         
                                                                              (ConnectionFactoryImpl.java:259)
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSourc         
                                                                              es(ConnectionFactoryImpl.java:131)
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.<init>(ConnectionFa   
                                                                                    ctoryImpl.java:85)
        ... 66 more
Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundExc         
                                                                              eption: The
specified datastore driver ("com.mysql.jdbc.Driver") was not found i                     
                                                                  n the CLASSPATH. Please
check your CLASSPATH specification, and the name of the                                  
                                                      driver.
        at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFact         
                                                                              ory.loadDriver(AbstractConnectionPoolFactory.java:58)
        at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactor         
                                                                              y.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources         
                                                                              (ConnectionFactoryImpl.java:238)
        ... 68 more

Thanks
Sandeep.v

On Mon, Mar 16, 2015 at 10:32 AM, Ted Yu <yuzhihong@gmail.com<mailto:yuzhihong@gmail.com>>
wrote:
Can you provide more information ?
Such as:
Version of Spark you're using
Command line

Thanks



> On Mar 15, 2015, at 9:51 PM, sandeep vura <sandeepvura@gmail.com<mailto:sandeepvura@gmail.com>>
wrote:
>
> Hi Sparkers,
>
>
>
> I couldn't able to run spark-sql on spark.Please find the following error
>
>  Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
>
>
> Regards,
> Sandeep.v



Mime
View raw message