hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1032) Better Error Messages for Execution Errors
Date Sat, 16 Jan 2010 16:03:54 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801202#action_12801202
] 

Namit Jain commented on HIVE-1032:
----------------------------------

I think the same error message can mean different things - it depends on the context.

Going forward, users will use more complex queries containing say a script and a join. 
If there is a out of memory exception (both join and the script operator) can do so, but if
the
script runs in the mapper and the join in the reducer, we can use that.

As you said that the operator can output different messages, the fact whether we are processing
a 
mapper of reducer log is also part of the same context. Today, we are processing all error
processors
for all the queries, but we can determine that at compile time (based on the query).


> Better Error Messages for Execution Errors
> ------------------------------------------
>
>                 Key: HIVE-1032
>                 URL: https://issues.apache.org/jira/browse/HIVE-1032
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Paul Yang
>            Assignee: Paul Yang
>         Attachments: HIVE-1032.1.patch, HIVE-1032.2.patch, HIVE-1032.3.patch
>
>
> Three common errors that occur during execution are:
> 1. Map-side group-by causing an out of memory exception due to large aggregation hash
tables
> 2. ScriptOperator failing due to the user's script throwing an exception or otherwise
returning a non-zero error code
> 3. Incorrectly specifying the join order of small and large tables, causing the large
table to be loaded into memory and producing an out of memory exception.
> These errors are typically discovered by manually examining the error log files of the
failed task. This task proposes to create a feature that would automatically read the error
logs and output a probable cause and solution to the command line.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message