hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13901) Hivemetastore add partitions can be slow depending on filesystems
Date Tue, 21 Jun 2016 00:07:57 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340743#comment-15340743
] 

Ashutosh Chauhan commented on HIVE-13901:
-----------------------------------------

Some of these tests are still failing to me when ran locally:
{code}
Running org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore
Tests run: 34, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 91.734 sec <<< FAILURE!
- in org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore
testPartition(org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore)  Time elapsed: 1.749
sec  <<< FAILURE!
junit.framework.AssertionFailedError: null
	at junit.framework.Assert.fail(Assert.java:55)
	at junit.framework.Assert.assertTrue(Assert.java:22)
	at junit.framework.Assert.assertTrue(Assert.java:31)
	at junit.framework.TestCase.assertTrue(TestCase.java:201)
	at org.apache.hadoop.hive.metastore.TestHiveMetaStore.partitionTester(TestHiveMetaStore.java:443)
	at org.apache.hadoop.hive.metastore.TestHiveMetaStore.testPartition(TestHiveMetaStore.java:146)
{code}
{code}
Running org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore
Tests run: 34, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 91.111 sec <<< FAILURE!
- in org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore
testTransactionalValidation(org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore)  Time
elapsed: 0.143 sec  <<< ERROR!
org.apache.hadoop.hive.metastore.api.AlreadyExistsException: Table acidTable already exists
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_with_environment_context_result$create_table_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41480)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_with_environment_context_result$create_table_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41466)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_with_environment_context_result.read(ThriftHiveMetastore.java:41392)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table_with_environment_context(ThriftHiveMetastore.java:1183)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table_with_environment_context(ThriftHiveMetastore.java:1169)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.create_table_with_environment_context(HiveMetaStoreClient.java:2325)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:738)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:726)
	at org.apache.hadoop.hive.metastore.TestHiveMetaStore.createTable(TestHiveMetaStore.java:2967)
	at org.apache.hadoop.hive.metastore.TestHiveMetaStore.testTransactionalValidation(TestHiveMetaStore.java:2897)
{code}

{code}
testPartition(org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore)  Time elapsed: 1.675
sec  <<< FAILURE!
junit.framework.AssertionFailedError: null
	at junit.framework.Assert.fail(Assert.java:55)
	at junit.framework.Assert.assertTrue(Assert.java:22)
	at junit.framework.Assert.assertTrue(Assert.java:31)
	at junit.framework.TestCase.assertTrue(TestCase.java:201)
	at org.apache.hadoop.hive.metastore.TestHiveMetaStore.partitionTester(TestHiveMetaStore.java:443)
	at org.apache.hadoop.hive.metastore.TestHiveMetaStore.testPartition(TestHiveMetaStore.java:146)
{code}
{code}
testPartition(org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer)  Time elapsed: 1.771
sec  <<< FAILURE!
junit.framework.AssertionFailedError: null
	at junit.framework.Assert.fail(Assert.java:55)
	at junit.framework.Assert.assertTrue(Assert.java:22)
	at junit.framework.Assert.assertTrue(Assert.java:31)
	at junit.framework.TestCase.assertTrue(TestCase.java:201)
	at org.apache.hadoop.hive.metastore.TestHiveMetaStore.partitionTester(TestHiveMetaStore.java:443)
	at org.apache.hadoop.hive.metastore.TestHiveMetaStore.testPartition(TestHiveMetaStore.java:146)
{code}

> Hivemetastore add partitions can be slow depending on filesystems
> -----------------------------------------------------------------
>
>                 Key: HIVE-13901
>                 URL: https://issues.apache.org/jira/browse/HIVE-13901
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Metastore
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>            Priority: Minor
>         Attachments: HIVE-13901.1.patch, HIVE-13901.2.patch, HIVE-13901.6.patch
>
>
> Depending on FS, creating external tables & adding partitions can be expensive (e.g
msck which adds all partitions).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message