hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ramki Palle <ramki.pa...@gmail.com>
Subject Re: java.lang.NoClassDefFoundError: com/jayway/jsonpath/PathUtil
Date Sun, 10 Mar 2013 11:22:00 GMT
When you execute the following query,

hive> select * from twitter limit 5;

Hive runs it in local mode and not use MapReduce.

For the query,

hive> select tweet_id from twitter limit 5;

I think you need to add JSON jars to overcome this error. You might have
added these in a previous session. If you want these jars available for all
sessions, insert the add jar statements to your $HOME/.hiverc file.


To bypass MapReduce

set hive.exec.mode.local.auto = true;

to suggest Hive to use local mode to execute the query. If it still uses
MR, try

set hive.fetch.task.conversion = more;.


-Ramki.



On Sun, Mar 10, 2013 at 12:19 AM, Sai Sai <saigraph@yahoo.in> wrote:

> Just wondering if anyone has any suggestions:
>
> This executes successfully:
>
> hive> select * from twitter limit 5;
>
> This does not work:
>
> hive> select tweet_id from twitter limit 5; // I have given the exception
> info below:
>
> Here is the output of this:
>
> hive> select * from twitter limit 5;
> OK
>
> tweet_id    created_at    text    user_id    user_screen_name    user_lang
> 122106088022745088    Fri Oct 07 00:28:54 +0000 2011    wkwkw -_- ayo saja
> mba RT @yullyunet: Sepupuuu, kita lanjalan yok.. Kita karokoe-an.. Ajak mas
> galih jg kalo dia mau.. "@Dindnf: doremifas    124735434    Dindnf    en
> 122106088018558976    Fri Oct 07 00:28:54 +0000 2011    @egg486 특별히
> 준비했습니다!    252828803    CocaCola_Korea    ko
> 122106088026939392    Fri Oct 07 00:28:54 +0000 2011    My offer of free
> gobbies for all if @amityaffliction play Blair snitch project still
> stands.    168590073    SarahYoungBlood    en
> 122106088035328001    Fri Oct 07 00:28:54 +0000 2011    the girl nxt to me
> in the lib got her headphones in dancing and singing loud af like she the
> only one here haha    267296295    MONEYyDREAMS_    en
> 122106088005971968    Fri Oct 07 00:28:54 +0000 2011    @KUnYoong_B2UTY
> Bị lsao đấy    269182160    b2st_b2utyhp    en
> Time taken: 0.154 seconds
>
> This does not work:
>
> hive> select tweet_id from twitter limit 5;
>
>
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_201303050432_0094, Tracking URL =
> http://ubuntu:50030/jobdetails.jsp?jobid=job_201303050432_0094
> Kill Command = /home/satish/work/hadoop-1.0.4/libexec/../bin/hadoop job
> -kill job_201303050432_0094
> Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 0
> 2013-03-10 00:14:44,509 Stage-1 map = 0%,  reduce = 0%
> 2013-03-10 00:15:14,613 Stage-1 map = 100%,  reduce = 100%
> Ended Job = job_201303050432_0094 with errors
> Error during job, obtaining debugging information...
> Job Tracking URL:
> http://ubuntu:50030/jobdetails.jsp?jobid=job_201303050432_0094
> Examining task ID: task_201303050432_0094_m_000002 (and more) from job
> job_201303050432_0094
>
> Task with the most failures(4):
> -----
> Task ID:
>   task_201303050432_0094_m_000000
>
> URL:
>
> http://ubuntu:50030/taskdetails.jsp?jobid=job_201303050432_0094&tipid=task_201303050432_0094_m_000000
> -----
> Diagnostic Messages for this Task:
> java.lang.RuntimeException: Error in configuring object
>     at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>     at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>     at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
>     at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:416)
>     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>     at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.reflect.InvocationTargetException
>
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:616)
>     at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>     ... 9 more
> Caused by: java.lang.RuntimeException: Error in configuring object
>     at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>     at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>     at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>     at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
>     ... 14 more
> Caused by: java.lang.reflect.InvocationTargetException
>
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:616)
>     at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>     ... 17 more
> Caused by: java.lang.RuntimeException: Map operator initialization failed
>     at
> org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
>     ... 22 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassNotFoundException:
> org.apache.hadoop.hive.contrib.serde2.JsonSerde
>     at
> org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
>     at
> org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
>     ... 22 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.hive.contrib.serde2.JsonSerde
>
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>     at java.lang.Class.forName0(Native Method)
>     at java.lang.Class.forName(Class.java:264)
>     at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
>     at
> org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:243)
>     at
> org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:380)
>     ... 23 more
>
>
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
> MapReduce Jobs Launched:
> Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
> Total MapReduce CPU Time Spent: 0 msec
>
> Thanks
> Sai
>   ------------------------------
> *From:* Dean Wampler <dean.wampler@thinkbiganalytics.com>
> *To:* user@hive.apache.org; Sai Sai <saigraph@yahoo.in>
> *Sent:* Friday, 8 March 2013 5:22 AM
> *Subject:* Re: java.lang.NoClassDefFoundError:
> com/jayway/jsonpath/PathUtil
>
> Unfortunately, you have to also add the json jars to Hive's class path
> before it starts, e.g.,
>
> env HADOOP_CLASSPATH=/path/to/lib/*.jar hive
>
> Use the appropriate path to your lib directory.
>
> On Fri, Mar 8, 2013 at 4:53 AM, Sai Sai <saigraph@yahoo.in> wrote:
>
> I have added the jar files successfully like this:
>
>
> hive (testdb)> ADD JAR lib/hive-json-serde-0.3.jar;
>                Added lib/hive-json-serde-0.3.jar to class path
>                Added resource: lib/hive-json-serde-0.3.jar
>
>
> hive (testdb)> ADD JAR lib/json-path-0.5.4.jar;
>                Added lib/json-path-0.5.4.jar to class path
>                Added resource: lib/json-path-0.5.4.jar
>
>
> hive (testdb)> ADD JAR lib/json-smart-1.0.6.3.jar;
>                Added lib/json-smart-1.0.6.3.jar to class path
>                Added resource: lib/json-smart-1.0.6.3.jar
>
>
> After this i am getting this error:
>
>
> CREATE EXTERNAL TABLE IF NOT EXISTS twitter (tweet_id BIGINT,created_at
> STRING,text STRING,user_id BIGINT, user_screen_name STRING,user_lang
> STRING) ROW FORMAT SERDE "org.apache.hadoop.hive.contrib.serde2.JsonSerde"
> WITH SERDEPROPERTIES (
> "tweet_id"="$.id","created_at"="$.created_at","text"="$.text","user_id"="$.
> user.id","user_screen_name"="$.user.screen_name",
> "user_lang"="$.user.lang") LOCATION '/home/satish/data/twitter/input';
> java.lang.NoClassDefFoundError: com/jayway/jsonpath/PathUtil
>     at org.apache.hadoop.hive.contrib.serde2.JsonSerde.initialize(Unknown
> Source)
>     at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:207)
>     at
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:266)
>     at
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:259)
>     at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:585)
>     at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:550)
>     at
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3698)
>     at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:253)
>     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
>     at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1336)
>     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1122)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:935)
>     at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)
>     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:616)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.ClassNotFoundException: com.jayway.jsonpath.PathUtil
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>     ... 23 more
> FAILED: Execution Error, return code -101 from
> org.apache.hadoop.hive.ql.exec.DDLTask
>
>
> Any help would be really appreciated.
> Thanks
> Sai
>
>
>
>
> --
> *Dean Wampler, Ph.D.*
> thinkbiganalytics.com
> +1-312-339-1330
>
>
>
>

Mime
View raw message