hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1032) Better Error Messages for Execution Errors
Date Thu, 14 Jan 2010 01:38:54 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800074#action_12800074
] 

Namit Jain commented on HIVE-1032:
----------------------------------

Had a discussion with Paul offline - it might be a good idea to separate the mapper logs and
reducer logs and then run possibly different heuristics on those.
For eg: if a query contains a join and a group by and no mapjoin, and a out of memory occurs
on the reducer, it is likely due to join order

> Better Error Messages for Execution Errors
> ------------------------------------------
>
>                 Key: HIVE-1032
>                 URL: https://issues.apache.org/jira/browse/HIVE-1032
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Paul Yang
>            Assignee: Paul Yang
>         Attachments: HIVE-1032.1.patch, HIVE-1032.2.patch
>
>
> Three common errors that occur during execution are:
> 1. Map-side group-by causing an out of memory exception due to large aggregation hash
tables
> 2. ScriptOperator failing due to the user's script throwing an exception or otherwise
returning a non-zero error code
> 3. Incorrectly specifying the join order of small and large tables, causing the large
table to be loaded into memory and producing an out of memory exception.
> These errors are typically discovered by manually examining the error log files of the
failed task. This task proposes to create a feature that would automatically read the error
logs and output a probable cause and solution to the command line.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message