hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Chu <e...@rocketfuel.com>
Subject java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.util.HostUtil for Hive 0.10.0
Date Wed, 27 Feb 2013 02:39:36 GMT
(+hue-user since this issue prevents me from successfully installing Hue
from source)

Hi,

I recently did the following with both the Hive-0.10 and Hive-0.9, and had
a problem with 0.10 that I didn't see with 0.9

   - Checked out the respective branch from github
   - Did an "ant package"
   - Copied the dist folder to /usr/lib/hive on the right machine
   - Copied mysql-connector-java-5.1.22-bin.jar to /usr/lib/hive/lib
   - Configured /etc/hive/conf (so same for both versions)

The problem is that when I use Hive-0.10, doing a "select count(1) from
table" (or anything that requires MR) would return a NoClassDefFound error
(see *Error Msg *below), whereas when I use Hive-0.9, the job would run
fine. Has anyone run into this problem? I can't use Hive-0.9 b/c it has
libthrift-0.7.0.jar, while Hue-2.2 (which I'm also using) requires
libthrift-0.9.0.jar (available in Hive-10).

Any insights would be much appreciated. Googling on this error doesn't get
very far. Thanks!

*Error Msg:*

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201302201756_0009, Tracking URL =
http://master-hadoop.pww-arp-dev.rfiserve.net:50030/jobdetails.jsp?jobid=job_201302201756_0009
Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_201302201756_0009
Hadoop job information for Stage-1: number of mappers: 1; number of
reducers: 1
2013-02-26 21:05:01,060 Stage-1 map = 0%,  reduce = 0%
2013-02-26 21:05:32,286 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201302201756_0009 with errors
Error during job, obtaining debugging information...
Job Tracking URL:
http://master-hadoop.pww-arp-dev.rfiserve.net:50030/jobdetails.jsp?jobid=job_201302201756_0009
Examining task ID: task_201302201756_0009_m_000002 (and more) from job
job_201302201756_0009
Exception in thread "Thread-29" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/util/HostUtil
 at
org.apache.hadoop.hive.shims.Hadoop23Shims.getTaskAttemptLogUrl(Hadoop23Shims.java:53)
at
org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.getTaskInfos(JobDebugger.java:186)
 at
org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.run(JobDebugger.java:142)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.mapreduce.util.HostUtil
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
 at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
 ... 4 more
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message