hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-19750) Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full acid
Date Sun, 03 Jun 2018 23:56:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499608#comment-16499608
] 

Eugene Koifman edited comment on HIVE-19750 at 6/3/18 11:55 PM:
----------------------------------------------------------------

This is very odd. Test bot reports a failure:
{noformat}
Regression
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01

Failing for the past 1 build (Since Failed#11481 )
Took 13 sec.
Error Message
{"writeid":0,"bucketid":536936448,"rowid":0} 1 2 file:/home/hiveptest/35.232.255.26-hiveptest-0/apache-github-source-source/ql/target/tmp/org.apache.hadoop.hive.ql.TestTxnCommands-1528061700968/warehouse/nonacidorctbl/base_10000001/bucket_00001
Stacktrace
java.lang.AssertionError: {"writeid":0,"bucketid":536936448,"rowid":0}	1	2	file:/home/hiveptest/35.232.255.26-hiveptest-0/apache-github-source-source/ql/target/tmp/org.apache.hadoop.hive.ql.TestTxnCommands-1528061700968/warehouse/nonacidorctbl/base_10000001/bucket_00001
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.hadoop.hive.ql.TestTxnCommands.testNonAcidToAcidConversion01(TestTxnCommands.java:915)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
but {{TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01}} does
not exist.
 {{TestTxnCommandsWithSplitUpdateAndVectorization extends TestTxnCommands2}} not {{TestTxnCommands}}

 

 

{{org.apache.hadoop.hive.ql.TestTxnCommands.testNonAcidToAcidConversion01}} does exist but
the 2 failure messages are identical except for the host name in the file path....


former: https://builds.apache.org/job/PreCommit-HIVE-Build/11481/testReport/org.apache.hadoop.hive.ql/TestTxnCommandsWithSplitUpdateAndVectorization/testNonAcidToAcidConversion01/

latter: https://builds.apache.org/job/PreCommit-HIVE-Build/11481/testReport/org.apache.hadoop.hive.ql/TestTxnCommands/testNonAcidToAcidConversion01/



was (Author: ekoifman):
This is very odd.  Test bot reports a failure:
{noformat}
Regression
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01

Failing for the past 1 build (Since Failed#11481 )
Took 13 sec.
Error Message
{"writeid":0,"bucketid":536936448,"rowid":0} 1 2 file:/home/hiveptest/35.232.255.26-hiveptest-0/apache-github-source-source/ql/target/tmp/org.apache.hadoop.hive.ql.TestTxnCommands-1528061700968/warehouse/nonacidorctbl/base_10000001/bucket_00001
Stacktrace
java.lang.AssertionError: {"writeid":0,"bucketid":536936448,"rowid":0}	1	2	file:/home/hiveptest/35.232.255.26-hiveptest-0/apache-github-source-source/ql/target/tmp/org.apache.hadoop.hive.ql.TestTxnCommands-1528061700968/warehouse/nonacidorctbl/base_10000001/bucket_00001
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.hadoop.hive.ql.TestTxnCommands.testNonAcidToAcidConversion01(TestTxnCommands.java:915)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}

but {{TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01}} does
not exist.
 {{TestTxnCommandsWithSplitUpdateAndVectorization extends TestTxnCommands2}} not {{TestTxnCommands}}

> Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full acid
> -------------------------------------------------------------------------------
>
>                 Key: HIVE-19750
>                 URL: https://issues.apache.org/jira/browse/HIVE-19750
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 3.0.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>            Priority: Critical
>             Fix For: 3.1.0
>
>         Attachments: HIVE-19750.01.patch, HIVE-19750.02.patch
>
>
> Need to set this to a reasonably high value the the table.
> This will reserve a range of write IDs that will be treated by the system as committed.
> This is needed so that we can assign unique ROW__IDs to each row in files that already
exist in the table.  For example, if the value is initialized to the number of files currently
in the table, we can think of each file as written by a separate transaction and thus a free
to assign bucketProperty (BucketCodec) of ROW_ID in whichever way is convenient.
> it's guaranteed that all rows get unique ROW_IDs this way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message