hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghotham Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-251) Failures in Transform don't stop the job
Date Tue, 03 Feb 2009 02:21:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669847#action_12669847
] 

Raghotham Murthy commented on HIVE-251:
---------------------------------------

more information about this error based on Venky's tests.

For the case where the job succeeds even though the script fails, the following logs appear:
--
2009-02-02 17:49:26,251 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: StreamThread ErrorProcessor
done
*2009-02-02 17:49:26,251 ERROR org.apache.hadoop.hive.ql.exec.ScriptOperator: Script failed
with code 1*
2009-02-02 17:49:26,251 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: StreamThread OutputProcessor
done
2009-02-02 17:49:26,251 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: DESERIALIZE_ERRORS:0
2009-02-02 17:49:26,252 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: SERIALIZE_ERRORS:0
2009-02-02 17:49:26,296 INFO org.apache.hadoop.mapred.TaskRunner: Task 'task_200901301729_7391_m_000000_0'
done.
--

For the case where the job fails:
--
2009-02-02 17:59:03,398 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: StreamThread ErrorProcessor
done
*2009-02-02 17:59:03,398 ERROR org.apache.hadoop.hive.ql.exec.ScriptOperator: Error in writing
to script: Broken pipe*
2009-02-02 17:59:03,398 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: StreamThread OutputProcessor
done
2009-02-02 17:59:03,399 INFO org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0
2009-02-02 17:59:03,399 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: SERIALIZE_ERRORS:0
2009-02-02 17:59:03,399 INFO org.apache.hadoop.hive.ql.exec.ScriptOperator: DESERIALIZE_ERRORS:0
2009-02-02 17:59:03,429 WARN org.apache.hadoop.mapred.TaskTracker: Error running child
java.lang.RuntimeException: java.io.IOException: Broken pipe
--

Looks like exceptions thrown in ScriptOperator.close are being ignored.

> Failures in Transform don't stop the job
> ----------------------------------------
>
>                 Key: HIVE-251
>                 URL: https://issues.apache.org/jira/browse/HIVE-251
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>            Reporter: S. Alex Smith
>
> If the program executed via a SELECT TRANSFORM() USING 'foo' exits with a non-zero exit
status, Hive proceeds as if nothing bad happened.  The main way that the user knows something
bad has happened is if the user checks the logs (probably because he got no output).  This
is doubly bad if the program only fails part of the time (say, on certain inputs) since the
job will still produce output and thus the problem will likely go undetected.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message