hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Time Less <timelessn...@gmail.com>
Subject Re: Bizarro Hive (Hadoop?) Error
Date Tue, 28 Jun 2011 17:40:34 GMT
I'm having a hard time interpreting the JIRA - it seems to be saying that
Hive is passing an incorrect "mapred.child.java.opts=XmxNNNM" parameter,
missing the - ... Is that correct? I could dig into the Hive source code and
look for this bit if that's so.

I admit, I'm a Java newbie, so I'm not entirely certain I understand how
Hive/Hadoop interact, but if Hive is passing incorrect memory arguments to
Hadoop tasks, and they're OOMing, then it seems the sort of fix I could
undertake successfully.


On Mon, Jun 27, 2011 at 5:13 PM, Sumanth V <vsumanth10@gmail.com> wrote:

> You are hitting this bug - https://issues.apache.org/jira/browse/HIVE-1579
> I consistently hit this bug for one of the Hive queries.
>
>
> Sumanth
>
>
>
> On Mon, Jun 27, 2011 at 5:08 PM, Time Less <timelessness@gmail.com> wrote:
>
>> Today I'm getting this error again. A Google search brought me back to...
>> you guessed it... my own post. But this time no HDFS corruption. Bounced all
>> services, namenode, jobtracker, datanodes, tasktrackers. Still same error.
>> Here's what it looks like:
>>
>> *Fsck Output:
>> *FSCK ended at Mon Jun 27 17:02:07 PDT 2011 in 818 milliseconds
>> The filesystem under path '/' is HEALTHY
>>
>> *Hive Query Output:
>> *-bash-3.2$ hive -e "select * from air_client_logs where loglevel =
>> '[FATAL]'"
>> Hive history file=/tmp/hdfs/hive_job_log_hdfs_201106271702_317571839.txt
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks is set to 0 since there's no reduce operator
>> Starting Job = job_201106271658_0002, Tracking URL =
>> http://hadooptest1:50030/jobdetails.jsp?jobid=job_201106271658_0002
>> Kill Command = /usr/lib/hadoop/bin/hadoop job
>> -Dmapred.job.tracker=hadooptest1:54311 -kill job_201106271658_0002
>> 2011-06-27 17:02:39,874 Stage-1 map = 0%,  reduce = 0%
>> 2011-06-27 17:03:02,111 Stage-1 map = 100%,  reduce = 100%
>> Ended Job = job_201106271658_0002 with errors
>>
>> java.lang.RuntimeException: Error while reading from task log url
>>     at
>> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
>>     at
>> org.apache.hadoop.hive.ql.exec.ExecDriver.showJobFailDebugInfo(ExecDriver.java:889)
>>     at
>> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:680)
>>     at
>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
>>     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
>>     at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
>>     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
>>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
>>     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
>>     at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
>>     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:425)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>>     at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>     at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>     at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>> Caused by: java.io.IOException: Server returned HTTP response code: 400
>> for URL:
>> http://hadooptest14:50060/tasklog?taskid=attempt_201106271658_0002_m_000008_0&all=true
>>
>>     at
>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
>>     at java.net.URL.openStream(URL.java:1010)
>>     at
>> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
>>     ... 16 more
>> Ended Job = job_201106271658_0002 with exception
>> 'java.lang.RuntimeException(Error while reading from task log url)'
>>
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>>
>> Again, trying to go to "
>> http://hadooptest14:50060/tasklog?taskid=attempt_201106271658_0002_m_000008_0&all=true"
>> returns that argument attemptid error.
>>
>> What am I doing wrong here? I appear to keep doing it, whatever it is.
>>
>>
>>
>> On Fri, May 6, 2011 at 6:47 PM, Time Less <timelessness@gmail.com> wrote:
>>
>>> My cluster went corrupt-mode. I wiped it and deleted the Hive metastore
>>> and started over. In the process, I did a "yum upgrade" which probably took
>>> me from CDH3b4 to CDH3u0. Now everytime I submit a Hive query of complexity
>>> requiring a map/reduce job*, I get this error:
>>>
>>> 2011-05-06 18:39:14,533 Stage-1 map = 100%,  reduce = 100%
>>> Ended Job = job_201104081532_0509 with errors
>>> java.lang.RuntimeException: Error while reading from task log url
>>>     at
>>> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
>>>     at
>>> org.apache.hadoop.hive.ql.exec.ExecDriver.showJobFailDebugInfo(ExecDriver.java:889)
>>>     at
>>> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:680)
>>>     at
>>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
>>> .....[snip].....
>>>     at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>     at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>     at java.lang.reflect.Method.invoke(Method.java:597)
>>>     at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>>> Caused by: java.io.IOException: Server returned HTTP response code: 400
>>> for URL:
>>> http://hadooptest3:50060/tasklog?taskid=attempt_201104081532_0509_m_000002_2&all=true
>>>     at
>>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
>>>     at java.net.URL.openStream(URL.java:1010)
>>>     at
>>> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
>>>     ... 16 more
>>> Ended Job = job_201104081532_0509 with exception
>>> 'java.lang.RuntimeException(Error while reading from task log url)'
>>> FAILED: Execution Error, return code 1 from
>>> org.apache.hadoop.hive.ql.exec.MapRedTask
>>>
>>> It seems to me the key point in here is this:
>>>
>>> Server returned HTTP response code: 400 for URL:
>>> http://hadooptest3:50060/tasklog?taskid=attempt_201104081532_0509_m_000002_2&all=true
>>>
>>> So I submitted that URL to my web browser, which said:
>>>
>>> Problem accessing /tasklog. Reason: Argument attemptid is required
>>>
>>> Does anyone have clues what this means?
>>>
>>> --
>>> Tim Ellis
>>> Riot Games
>>> * That is to say, simple queries like "select * from table limit 10"
>>> return results just fine.
>>>
>>>
>>
>>
>> --
>> Tim
>>
>
>


-- 
Tim

Mime
View raw message