hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sal Scalisi <sal...@hotmail.com>
Subject Re: Problem With Simple Select From Hive
Date Thu, 07 Jul 2011 20:04:42 GMT
I have some more information that may shed some light on this issue.  I 
re-ran my query with hive configured to log to the console window.  I 
noticed the following error from AWS:

11/07/07 19:56:42 INFO amazonaws.request: Sending Request: HEAD 
https://mapreduce-dev.s3.amazonaws.com /log_parser%2Finput_%24folder%24 
Headers: (Authorization: AWS XXXX=, Date: Thu, 07 Jul 2011 19:56:42 GMT, 
Content-Type: application/x-www-form-urlencoded; charset=utf-8, )
11/07/07 19:56:42 INFO amazonaws.request: Received error response: 
Status Code: 404, AWS Request ID: XXXX, AWS Error Code: null, AWS Error 
Message: Not Found, S3 Extended Request ID:

I've tried to make my S3 folders public, but that did not resolve the 
issue.  If anyone has any insight, it is appreciated.

-Sal

On 7/6/2011 3:12 PM, Sal Scalisi wrote:
I'm teaching myself hive and I'm having an issue with a simple select 
statement.

I'm using Amazon's Elastic MapReduce.

I'm creating a table with some sample web log data I downloaded from a 
sample given by Amazon.

I create this table:
CREATE EXTERNAL TABLE log_parser_stage (
       remoteAddr    STRING,
       remoteLogname STRING,
       user          STRING,
       time          STRING,
       request       STRING,
       status        STRING,
       bytes_string  STRING,
       referrer      STRING,
       browser       STRING
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
   "input.regex" = '^(\\S+) (\\S+) (\\S+) 
\\[([\\w:/]+\\s[+\\-]\\d{4})\\] "(.+?)" (\\S+) (\\S+) "([^"]*)" "([^"]*)"',
   "output.format.string" = "%1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s")
LOCATION 's3://mys3bucket/';

I then run a simple select * from the table to insure there is data there:

hive> SELECT * FROM log_parser_stage LIMIT 10;
OK
66.249.67.3     -       -       20/Jul/2009:20:12:22 -0700      GET 
/gallery/main.php?g2_controller=exif.SwitchDetailMode&g2_mode=detailed&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D15741&g2_returnName=photo

HTTP/1.1   302     5       -       Mozilla/5.0 (compatible; 
Googlebot/2.1; +http://www.google.com/bot.html)
66.249.67.3     -       -       20/Jul/2009:20:12:25 -0700      GET 
/gallery/main.php?g2_itemId=15741&g2_fromNavId=x8fa12efc HTTP/1.1   
200     8068    -       Mozilla/5.0 (compatible; Googlebot/2.1; 
+http://www.google.com/bot.html)
64.233.172.17   -       -       20/Jul/2009:20:12:26 -0700      GET 
/gwidgets/alexa.xml HTTP/1.1        200     2969    -       Mozilla/5.0 
(compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)
74.125.74.193   -       -       20/Jul/2009:20:13:01 -0700      GET 
/gwidgets/alexa.xml HTTP/1.1        200     2969    -       Mozilla/5.0 
(compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)
192.168.1.198   -       -       20/Jul/2009:20:13:18 -0700      GET / 
HTTP/1.1  200     17935   -       Mozilla/5.0 (Macintosh; U; Intel Mac 
OS X 10_5_7; en-us) AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0 
Safari/530.17
192.168.1.198   -       -       20/Jul/2009:20:13:18 -0700      GET 
/style.css HTTP/1.1 200     1504    http://example.org/     Mozilla/5.0 
(Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/530.17 (KHTML, 
like Gecko) Version/4.0 Safari/530.17
192.168.1.198   -       -       20/Jul/2009:20:13:19 -0700      GET 
/favicon.ico HTTP/1.1       404     146     http://example.org/     
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us) 
AppleWebKit/530.17 (KHTML, like Gecko) Version/4.0 Safari/530.17
66.249.67.3     -       -       20/Jul/2009:20:13:21 -0700      GET 
/gallery/main.php?g2_controller=exif.SwitchDetailMode&g2_mode=detailed&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D30893&g2_returnName=photo

HTTP/1.1   302     5       -       Mozilla/5.0 (compatible; 
Googlebot/2.1; +http://www.google.com/bot.html)
66.249.67.3     -       -       20/Jul/2009:20:13:24 -0700      GET 
/gallery/main.php?g2_itemId=30893&g2_fromNavId=xfc647d65 HTTP/1.1   
200     8196    -       Mozilla/5.0 (compatible; Googlebot/2.1; 
+http://www.google.com/bot.html)
66.249.67.3     -       -       20/Jul/2009:20:13:29 -0700      GET 
/gallery/main.php?g2_view=search.SearchScan&g2_form%5BuseDefaultSettings%5D=1&g2_return=%2Fgallery%2Fmain.php%3Fg2_itemId%3D15789&g2_returnName=photo

HTTP/1.1    200     6360    -       Mozilla/5.0 (compatible; 
Googlebot/2.1; +http://www.google.com/bot.html)
Time taken: 13.954 seconds

I then try to query for two fields:

hive> SELECT remoteAddr, browser FROM log_parser_stage LIMIT 10;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201107062035_0007, Tracking URL = 
http://ip-10-36-79-98.ec2.internal:9100/jobdetails.jsp?jobid=job_201107062035_0007 

Kill Command = /home/hadoop/.versions/0.20/bin/../bin/hadoop job  
-Dmapred.job.tracker=ip-10-36-79-98.ec2.internal:9001 -kill 
job_201107062035_0007
2011-07-06 22:06:21,137 Stage-1 map = 0%,  reduce = 0%
2011-07-06 22:07:05,974 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201107062035_0007 with errors

Failed tasks with most(4) failures :
Task URL: 
http://ip-10-36-79-98.ec2.internal:9100/taskdetails.jsp?jobid=job_201107062035_0007&tipid=task_201107062035_0007_m_000000



FAILED: Execution Error, return code 2 from 
org.apache.hadoop.hive.ql.exec.ExecDriver

The following is from the job logs:

Meta VERSION="1" .
Job JOBID="job_201107062035_0006" JOBNAME="SELECT remoteAddr, 
browse\.\.\.log_parser_stage(Stage-1)" USER="hadoop" 
SUBMIT_TIME="1309989443667" 
JOBCONF="hdfs://ip-10-36-79-98\.ec2\.internal:9000/mnt/var/lib/hadoop/tmp/mapred/system/job_201107062035_0006/job\.xml"

.
Job JOBID="job_201107062035_0006" JOB_PRIORITY="NORMAL" .
Job JOBID="job_201107062035_0006" LAUNCH_TIME="1309989444146" 
TOTAL_MAPS="1" TOTAL_REDUCES="0" JOB_STATUS="PREP" .
Task TASKID="task_201107062035_0006_m_000002" TASK_TYPE="SETUP" 
START_TIME="1309989444665" SPLITS="" .
MapAttempt TASK_TYPE="SETUP" TASKID="task_201107062035_0006_m_000002" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000002_0" 
START_TIME="1309989445566" 
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"

HTTP_PORT="9103" .
MapAttempt TASK_TYPE="SETUP" TASKID="task_201107062035_0006_m_000002" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000002_0" 
TASK_STATUS="SUCCESS" FINISH_TIME="1309989447674" 
HOSTNAME="/default-rack/ip-10-36-79-98\.ec2\.internal" 
STATE_STRING="setup" 
COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce 
Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
Task TASKID="task_201107062035_0006_m_000002" TASK_TYPE="SETUP" 
TASK_STATUS="SUCCESS" FINISH_TIME="1309989450674" 
COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce 
Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
Job JOBID="job_201107062035_0006" JOB_STATUS="RUNNING" .
Task TASKID="task_201107062035_0006_m_000000" TASK_TYPE="MAP" 
START_TIME="1309989450675" SPLITS="/default-rack/localhost" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_0" 
START_TIME="1309989450701" 
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"

HTTP_PORT="9103" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_0" 
TASK_STATUS="FAILED" FINISH_TIME="1309989457869" 
HOSTNAME="ip-10-36-79-98\.ec2\.internal" 
ERROR="java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
     at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
     at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
     \.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
     \.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException: 
java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
     \.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
     at java\.security\.AccessController\.doPrivileged(Native Method)
     at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
     at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
     at java\.lang\.Class\.forName0(Native Method)
     at java\.lang\.Class\.forName(Class\.java:247)
     at 
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
     \.\.\. 19 more
" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_1" 
START_TIME="1309989459732" 
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"

HTTP_PORT="9103" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_1" 
TASK_STATUS="FAILED" FINISH_TIME="1309989467109" 
HOSTNAME="ip-10-36-79-98\.ec2\.internal" 
ERROR="java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
     at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
     at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
     \.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
     \.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException: 
java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
     \.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
     at java\.security\.AccessController\.doPrivileged(Native Method)
     at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
     at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
     at java\.lang\.Class\.forName0(Native Method)
     at java\.lang\.Class\.forName(Class\.java:247)
     at 
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
     \.\.\. 19 more
" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_2" 
START_TIME="1309989468821" 
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"

HTTP_PORT="9103" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_2" 
TASK_STATUS="FAILED" FINISH_TIME="1309989477378" 
HOSTNAME="ip-10-36-79-98\.ec2\.internal" 
ERROR="java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
     at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
     at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
     \.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
     \.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException: 
java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
     \.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
     at java\.security\.AccessController\.doPrivileged(Native Method)
     at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
     at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
     at java\.lang\.Class\.forName0(Native Method)
     at java\.lang\.Class\.forName(Class\.java:247)
     at 
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
     \.\.\. 19 more
" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_3" 
START_TIME="1309989477905" 
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"

HTTP_PORT="9103" .
MapAttempt TASK_TYPE="MAP" TASKID="task_201107062035_0006_m_000000" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000000_3" 
TASK_STATUS="FAILED" FINISH_TIME="1309989484902" 
HOSTNAME="ip-10-36-79-98\.ec2\.internal" 
ERROR="java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
     at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
     at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
     \.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
     \.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException: 
java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
     \.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
     at java\.security\.AccessController\.doPrivileged(Native Method)
     at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
     at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
     at java\.lang\.Class\.forName0(Native Method)
     at java\.lang\.Class\.forName(Class\.java:247)
     at 
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
     \.\.\. 19 more
" .
Task TASKID="task_201107062035_0006_m_000000" TASK_TYPE="MAP" 
TASK_STATUS="FAILED" FINISH_TIME="1309989484902" 
ERROR="java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:359)
     at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:312)
     at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:170)
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 5 more
Caused by: java\.lang\.RuntimeException: Error in configuring object
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:93)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setConf(ReflectionUtils\.java:64)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.newInstance(ReflectionUtils\.java:117)
     at 
org\.apache\.hadoop\.mapred\.MapRunner\.configure(MapRunner\.java:34)
     \.\.\. 10 more
Caused by: java\.lang\.reflect\.InvocationTargetException
     at sun\.reflect\.NativeMethodAccessorImpl\.invoke0(Native Method)
     at 
sun\.reflect\.NativeMethodAccessorImpl\.invoke(NativeMethodAccessorImpl\.java:39)
     at 
sun\.reflect\.DelegatingMethodAccessorImpl\.invoke(DelegatingMethodAccessorImpl\.java:25)
     at java\.lang\.reflect\.Method\.invoke(Method\.java:597)
     at 
org\.apache\.hadoop\.util\.ReflectionUtils\.setJobConf(ReflectionUtils\.java:88)
     \.\.\. 13 more
Caused by: java\.lang\.RuntimeException: Map operator initialization failed
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:110)
     \.\.\. 18 more
Caused by: org\.apache\.hadoop\.hive\.ql\.metadata\.HiveException: 
java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:291)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.ExecMapper\.configure(ExecMapper\.java:80)
     \.\.\. 18 more
Caused by: java\.lang\.ClassNotFoundException: 
org\.apache\.hadoop\.hive\.contrib\.serde2\.RegexSerDe
     at java\.net\.URLClassLoader$1\.run(URLClassLoader\.java:202)
     at java\.security\.AccessController\.doPrivileged(Native Method)
     at java\.net\.URLClassLoader\.findClass(URLClassLoader\.java:190)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:307)
     at sun\.misc\.Launcher$AppClassLoader\.loadClass(Launcher\.java:301)
     at java\.lang\.ClassLoader\.loadClass(ClassLoader\.java:248)
     at java\.lang\.Class\.forName0(Native Method)
     at java\.lang\.Class\.forName(Class\.java:247)
     at 
org\.apache\.hadoop\.conf\.Configuration\.getClassByName(Configuration\.java:778)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.initObjectInspector(MapOperator\.java:181)
     at 
org\.apache\.hadoop\.hive\.ql\.exec\.MapOperator\.setChildren(MapOperator\.java:249)
     \.\.\. 19 more
" TASK_ATTEMPT_ID="" .
Task TASKID="task_201107062035_0006_m_000001" TASK_TYPE="CLEANUP" 
START_TIME="1309989486882" SPLITS="" .
MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201107062035_0006_m_000001" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000001_0" 
START_TIME="1309989486898" 
TRACKER_NAME="tracker_ip-10-36-79-98\.ec2\.internal:localhost\.localdomain/127\.0\.0\.1:51327"

HTTP_PORT="9103" .
MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201107062035_0006_m_000001" 
TASK_ATTEMPT_ID="attempt_201107062035_0006_m_000001_0" 
TASK_STATUS="SUCCESS" FINISH_TIME="1309989489057" 
HOSTNAME="/default-rack/ip-10-36-79-98\.ec2\.internal" 
STATE_STRING="cleanup" 
COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce 
Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
Task TASKID="task_201107062035_0006_m_000001" TASK_TYPE="CLEANUP" 
TASK_STATUS="SUCCESS" FINISH_TIME="1309989489906" 
COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce 
Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" .
Job JOBID="job_201107062035_0006" FINISH_TIME="1309989489907" 
JOB_STATUS="FAILED" FINISHED_MAPS="0" FINISHED_REDUCES="0" .

Any help is appreciated.

-Sal

Mime
View raw message