hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-873) Better error messages for Hive cmdline
Date Thu, 22 Oct 2009 01:04:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768539#action_12768539
] 

Zheng Shao commented on HIVE-873:
---------------------------------

bq. Why was rj.getTaskCompletionEvents() was checked repeatedly in a loop? I removed the loop
in the revision.

We only get 10 TaskCompletionEvent for each call - that's why we need to do it in a loop.

I agree the number 10 is not a good number - we should add the parameter to RunningJob.getTaskCompletionEvents()
but that won't happen soon since it's an Interface change.
Internally we can change 10 in the hadoop code to 100 to make it more efficient.

{code}
JobClient.NetworkedJob:
    /**
     * Fetch task completion events from jobtracker for this job. 
     */
    public synchronized TaskCompletionEvent[] getTaskCompletionEvents(
                                                                      int startFrom) throws
IOException{
      return jobSubmitClient.getTaskCompletionEvents(
                                                     getID(), startFrom, 10); 
    }
{code}


bq. Is there a better way to get the URL for the task info page? conf.get("mapred.job.tracker",
"local") has the wrong port # and TaskCompletionEvent.getTaskTrackerHttp() doesn't return
a usable address.

I remember in "Throttle.java" we have some similar code. I think we should extract it out
to a single function so it's easier to change in the future (when hadoop provides that). That
might be the best option for us now till we find a better one.

bq. Is the conf var hive.exec.fail.debug necessary? Should such info always be displayed?
I agree we can get rid of this option and make it always happen.



> Better error messages for Hive cmdline
> --------------------------------------
>
>                 Key: HIVE-873
>                 URL: https://issues.apache.org/jira/browse/HIVE-873
>             Project: Hadoop Hive
>          Issue Type: New Feature
>            Reporter: Zheng Shao
>            Assignee: Paul Yang
>         Attachments: hive-873-2009-10-15.patch, HIVE-873.1.patch
>
>
> Currently, we are facing a lot of questions of Hive users because they cannot directly
see what is the error causing the job to fail.
> We should expose that information to our user so that they can understand why their job
failed easily.
> There are the 2 possible ways to get the error information:
> {code}
> RunningJob JobClient.submitJob();
> TaskCompletionEvent[] RunningJob.getTaskCompletionEvents()
> TaskCompletionEvent.getTaskTrackerHttp()
> {code}
> {code}
> TaskReport[] JobClient.getMapTaskReports();
> {code}
> We can edit ExecDriver.java and put that in after we submit a job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message