hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1032) Better Error Messages for Execution Errors
Date Sat, 16 Jan 2010 23:57:54 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801299#action_12801299
] 

Zheng Shao commented on HIVE-1032:
----------------------------------

Whether it's in the mapper or reducer is not useful by itself - we will have to hook ErrorHeuristics
with Operator Tree to find it out.

Instead of going that route, it will be much easier for the ErrorHeuristics capture all information
from the log. We already have operator initialization messages in the log, so ErrorHeuristics
should already know what are the possible operators that caused the problem.

In this way, we can make the ErrorHeuristics loosely coupled with the compilation/query execution.


> Better Error Messages for Execution Errors
> ------------------------------------------
>
>                 Key: HIVE-1032
>                 URL: https://issues.apache.org/jira/browse/HIVE-1032
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Paul Yang
>            Assignee: Paul Yang
>         Attachments: HIVE-1032.1.patch, HIVE-1032.2.patch, HIVE-1032.3.patch
>
>
> Three common errors that occur during execution are:
> 1. Map-side group-by causing an out of memory exception due to large aggregation hash
tables
> 2. ScriptOperator failing due to the user's script throwing an exception or otherwise
returning a non-zero error code
> 3. Incorrectly specifying the join order of small and large tables, causing the large
table to be loaded into memory and producing an out of memory exception.
> These errors are typically discovered by manually examining the error log files of the
failed task. This task proposes to create a feature that would automatically read the error
logs and output a probable cause and solution to the command line.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message