hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-9861) Invert ReflectionUtils' stack trace
Date Sat, 10 Aug 2013 07:56:48 GMT
Harsh J created HADOOP-9861:

             Summary: Invert ReflectionUtils' stack trace
                 Key: HADOOP-9861
                 URL: https://issues.apache.org/jira/browse/HADOOP-9861
             Project: Hadoop Common
          Issue Type: Improvement
          Components: util
    Affects Versions: 2.0.5-alpha
            Reporter: Harsh J

Often an MR task (as an example) may fail at the configure stage due to a misconfiguration
or whatever, and the only thing a user gets by virtue of MR pulling limited bytes of the diagnostic
error data is the top part of the stacktrace:

java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

This is absolutely useless to a user, and he also goes ahead and blames the framework for
having an issue, rather than thinking (non-intuitively) to go see the whole task log for the
full trace, especially the last part.

Hundreds of time its been a mere class thats missing, etc. but there's just too much pain
involved here to troubleshoot.

Would be much much better, if we inverted the trace. For example, here's what Hive can return
back if we did so, for a random trouble I pulled from the web:

java.lang.RuntimeException: Error in configuring object
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector.toString(StructObjectInspector.java:64)
    at java.lang.String.valueOf(String.java:2826)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at org.apache.hadoop.hive.ql.exec.UnionOperator.initializeOp(UnionOperator.java:110)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
    at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
    at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:563)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100)
    ... 22 more

This way the user can at least be sure what part's really failing, and not get lost trying
to work their way through reflection utils and upwards/downwards.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message