hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (Jira)" <>
Subject [jira] [Commented] (HIVE-22393) HiveStreamingConnection: Exception in beginTransaction causes AbstractRecordWriter to throw NPE, covering up real exception
Date Fri, 24 Jan 2020 13:28:00 GMT


Hive QA commented on HIVE-22393:

Here are the results of testing the latest attachment:

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17936 tests executed
*Failed tests:*
TestJdbcWithMiniLlapArrow - did not produce a TEST-*.xml file (likely timed out) (batchId=288)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testNoBucketsDP (batchId=340)

Test results:
Console output:
Test logs:

Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed

This message is automatically generated.

ATTACHMENT ID: 12991694 - PreCommit-HIVE-Build

> HiveStreamingConnection: Exception in beginTransaction causes AbstractRecordWriter to
throw NPE, covering up real exception
> ---------------------------------------------------------------------------------------------------------------------------
>                 Key: HIVE-22393
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 3.1.2
>            Reporter: Alex Parrill
>            Assignee: Matt Burgess
>            Priority: Major
>         Attachments: HIVE-22393.1.patch
> Getting an error in my code that does some basic stuff with {{HiveStreamingConnection}}:
> {code}
> 19/10/23 15:12:44 ERROR yarn_logger.Main: Thread worker-0 uncaught exception
> java.lang.RuntimeException: org.apache.hive.streaming.StreamingException: Unable to close
> 	at
> Caused by: org.apache.hive.streaming.StreamingException: Unable to close
> 	at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(
> 	at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.markDead(
> 	at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.<init>(
> 	at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.<init>(
> 	at org.apache.hive.streaming.HiveStreamingConnection.createNewTransactionBatch(
> 	at org.apache.hive.streaming.HiveStreamingConnection.beginNextTransaction(
> 	at org.apache.hive.streaming.HiveStreamingConnection.beginTransaction(
> 	at
> Caused by: java.lang.NullPointerException
> 	at org.apache.hive.streaming.AbstractRecordWriter.logStats(
> 	at org.apache.hive.streaming.AbstractRecordWriter.close(
> 	at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.closeImpl(
> 	at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.close(
> 	... 7 more
> {code}
> Digging through the stack trace... {{TransactionBatch}} will try to catch exception in
its constructor, and calls the close method if an exception is thrown (which is definitely
happening; that's line 677 in {{}}). This eventually calls {{AbstractRecordWriter::close}}
which, calls {{logStats}}, which tries to use {{heapMemoryMonitor}}, which is null because
presumably {{init}} had never been called on the writer yet.
> Easy fix would to just check in {{logStats}} if {{heapMemoryMonitor}} is null, and do
the same thing if the method it calls returns null in it.

This message was sent by Atlassian Jira

View raw message