hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin BONNET (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-18412) FileSinkOperator thows NullPointerException
Date Tue, 09 Jan 2018 09:40:00 GMT
Benjamin BONNET created HIVE-18412:
--------------------------------------

             Summary: FileSinkOperator thows NullPointerException 
                 Key: HIVE-18412
                 URL: https://issues.apache.org/jira/browse/HIVE-18412
             Project: Hive
          Issue Type: Bug
          Components: Hive
         Environment: HDP2.6.1, Hive 1.2.1
            Reporter: Benjamin BONNET
            Priority: Blocker


Hi,
while executing a query (DELETE with a join) on an ACID table, I get a NullPointerException
in reducer.
See stack trace below.
According to FileSinkOperator source code, it seems that buckepMap transient field is Null.
In my opinion, the only circumstance in which this field may be null is when the involved
FileSinkOperator has been serialized and then deserialized. Actually, deserialization lets
that transient reference uninitialized.
I checked source code for more recent versions (including Hive 2.x) but everywhere that field
may remain uninitialized (if FileSinkOperator is serialized/deserialized). So I think that
issue may concern any version of Hive.

ERROR : Vertex failed, vertexName=Reducer 3, vertexId=vertex_1513704146031_77754_2_05, diagnostics=[Task
failed, taskId=task_1513704146031_77754_2_05_000000, diagnostics=[TaskAttempt 0 failed, info=[Error:
Failure while running task:java
.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"
_col0":"2017","_col1":"10"}}
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
        at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive
Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"_col0":"2017"
,"_col1":"10"}}
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:284)
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:266)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
        ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing
row (tag=0) {"key":{"reducesinkkey0":{"transactionid":108117,"bucketid":0,"rowid":1114}},"value":{"_col0":"2017","_col1":"10"}}
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
        ... 16 more
Caused by: java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.findWriterOffset(FileSinkOperator.java:830)
        at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:758)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:841)
        at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
        ... 17 more
], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException:
java.lang.RuntimeException: .... etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message