hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14009) Acid DB creation error in HiveQA
Date Tue, 14 Jun 2016 22:27:29 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330767#comment-15330767
] 

Eugene Koifman commented on HIVE-14009:
---------------------------------------

[~spena] could you comment?

> Acid DB creation error in HiveQA
> --------------------------------
>
>                 Key: HIVE-14009
>                 URL: https://issues.apache.org/jira/browse/HIVE-14009
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>
> Seen when running TestEncryptedHDFSCliDriver, at least with Hadoop 2.7.2 (HIVE-13930).

> Looks like such issues are usually caused by concurrent db creation from multiple threads.
> {noformat}
> java.lang.RuntimeException: Unable to set up transaction database for testing: Exception
during creation of file /home/hiveptest/54.219.24.101-hiveptest-0/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db/seg0/cc60.dat
for container
> 	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkQFileTestHack(TxnHandler.java:2172)
~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.txn.TxnHandler.setConf(TxnHandler.java:228) ~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.txn.TxnUtils.getTxnStore(TxnUtils.java:96) [hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTxnHandler(HiveMetaStore.java:557)
[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.heartbeat(HiveMetaStore.java:5902)
[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_25]
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_25]
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_25]
> 	at java.lang.reflect.Method.invoke(Method.java:483) ~[?:1.8.0_25]
> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at com.sun.proxy.$Proxy111.heartbeat(Unknown Source) [?:?]
> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.heartbeat(HiveMetaStoreClient.java:2140)
[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_25]
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_25]
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_25]
> 	at java.lang.reflect.Method.invoke(Method.java:483) ~[?:1.8.0_25]
> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:154)
[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at com.sun.proxy.$Proxy112.heartbeat(Unknown Source) [?:?]
> 	at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$SynchronizedMetaStoreClient.heartbeat(DbTxnManager.java:663)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.heartbeat(DbTxnManager.java:423) [hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$Heartbeater.run(DbTxnManager.java:633)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_25]
> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_25]
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_25]
> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[?:1.8.0_25]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_25]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_25]
> 	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_25]
> Caused by: java.sql.SQLException: Exception during creation of file /home/hiveptest/54.219.24.101-hiveptest-0/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db/seg0/cc60.dat
for container
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.hadoop.hive.metastore.txn.TxnDbUtil.prepDb(TxnDbUtil.java:75) ~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkQFileTestHack(TxnHandler.java:2167)
~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	... 28 more
> Caused by: org.apache.derby.impl.jdbc.EmbedSQLException: Exception during creation of
file /home/hiveptest/54.219.24.101-hiveptest-0/apache-github-source-source/itests/qtest/target/tmp/junit_metastore_db/seg0/cc60.dat
for container
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.hadoop.hive.metastore.txn.TxnDbUtil.prepDb(TxnDbUtil.java:75) ~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkQFileTestHack(TxnHandler.java:2167)
~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	... 28 more
> Caused by: org.apache.derby.impl.jdbc.EmbedSQLException: Java exception: 'sync failed:
java.io.SyncFailedException'.
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.hadoop.hive.metastore.txn.TxnDbUtil.prepDb(TxnDbUtil.java:75) ~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkQFileTestHack(TxnHandler.java:2167)
~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	... 28 more
> Caused by: java.io.SyncFailedException: sync failed
> 	at java.io.FileDescriptor.sync(Native Method) ~[?:1.8.0_25]
> 	at org.apache.derby.impl.io.DirRandomAccessFile.sync(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.raw.data.RAFContainer.writeRAFHeader(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.raw.data.RAFContainer.run(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_25]
> 	at org.apache.derby.impl.store.raw.data.RAFContainer.createContainer(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.raw.data.RAFContainer4.createContainer(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.raw.data.FileContainer.createIdent(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.raw.data.FileContainer.createIdentity(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.services.cache.ConcurrentCache.create(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.addContainer(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.raw.xact.Xact.addContainer(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.access.heap.Heap.create(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.access.heap.HeapConglomerateFactory.createConglomerate(Unknown
Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.store.access.RAMTransaction.createConglomerate(Unknown Source)
~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.sql.execute.CreateTableConstantAction.executeConstantAction(Unknown
Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.sql.execute.MiscResultSet.open(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source) ~[derby-10.10.2.0.jar:?]
> 	at org.apache.hadoop.hive.metastore.txn.TxnDbUtil.prepDb(TxnDbUtil.java:75) ~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkQFileTestHack(TxnHandler.java:2167)
~[hive-metastore-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> 	... 28 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message