hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Zhang" <nzh...@fb.com>
Subject Re: Review Request: HIVE-2156: Improve Execution Error Messages
Date Tue, 24 May 2011 20:49:24 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/777/#review711
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java
<https://reviews.apache.org/r/777/#comment1420>

    error code -101 is also used in TaskRunner.java to indicate OOM exception. We should define
all these error code in a centralized place.  



ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment1421>

    We should use interface in the declaration (Set<String>). 



ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment1422>

    Also the return type should use interface rather than class implementation. 



ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment1423>

    Print out error message related to the exception so that user know what goes wrong?



ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment1424>

    Some task log are very big. retrieving all may take a long time. It may be sufficient
to just retrieve the last 8KB. 



ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java
<https://reviews.apache.org/r/777/#comment1425>

    Do you have some numbers on how long it takes to get all the TaskCompletionEvents? There
are cases that a job may have more than 10k tasks and all of them failed with the same error.
    
    If it takes too long you may want to consider adding a threshold to the time spent in
getting all the TaskCompleteEvents. 


- Ning


On 2011-05-24 04:29:32, Syed Albiz wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/777/
> -----------------------------------------------------------
> 
> (Updated 2011-05-24 04:29:32)
> 
> 
> Review request for hive and John Sichi.
> 
> 
> Summary
> -------
> 
> - Add local error messages to point to job logs and provide TaskIDs
> - Add a timeout to the fetching of task logs and errors
> 
> 
> This addresses bug HIVE-2156.
>     https://issues.apache.org/jira/browse/HIVE-2156
> 
> 
> Diffs
> -----
> 
>   build-common.xml 00c3680 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java dc96a1f 
>   conf/hive-default.xml 159d825 
>   ql/build.xml 449b47a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java 4717c25 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/JobDebugger.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java 53769a0 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java 691f038 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 9cb407c 
>   ql/src/test/queries/clientnegative/minimr_broken_pipe.q PRE-CREATION 
>   ql/src/test/results/clientnegative/dyn_part3.q.out 5f4df65 
>   ql/src/test/results/clientnegative/minimr_broken_pipe.q.out PRE-CREATION 
>   ql/src/test/results/clientnegative/script_broken_pipe1.q.out d33d2cc 
>   ql/src/test/results/clientnegative/script_broken_pipe2.q.out afbaa44 
>   ql/src/test/results/clientnegative/script_broken_pipe3.q.out fe8f757 
>   ql/src/test/results/clientnegative/script_error.q.out c72d780 
>   ql/src/test/results/clientnegative/udf_reflect_neg.q.out f2082a3 
>   ql/src/test/results/clientnegative/udf_test_error.q.out 5fd9a00 
>   ql/src/test/results/clientnegative/udf_test_error_reduce.q.out ddc5e5b 
>   ql/src/test/templates/TestNegativeCliDriver.vm ec13f79 
> 
> Diff: https://reviews.apache.org/r/777/diff
> 
> 
> Testing
> -------
> 
> Tested TestNegativeCliDriver in both local and miniMR mode
> 
> 
> Thanks,
> 
> Syed
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message