hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16450) Some metastore operations are not retried even with desired underlining exceptions
Date Sat, 06 May 2017 14:23:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999445#comment-15999445
] 

Hive QA commented on HIVE-16450:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12866624/HIVE-16450.4.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 10652 tests passed

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5076/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5076/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5076/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12866624 - PreCommit-HIVE-Build

> Some metastore operations are not retried even with desired underlining exceptions
> ----------------------------------------------------------------------------------
>
>                 Key: HIVE-16450
>                 URL: https://issues.apache.org/jira/browse/HIVE-16450
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 2.0.0
>            Reporter: Aihua Xu
>            Assignee: Aihua Xu
>         Attachments: HIVE-16450.1.patch, HIVE-16450.2.patch, HIVE-16450.3.patch, HIVE-16450.4.patch
>
>
> In RetryingHMSHandler class, we are expecting the operations should retry when the cause
of MetaException is JDOException or NucleusException.
> {noformat}
>         if (e.getCause() instanceof MetaException && e.getCause().getCause()
!= null) {
>           if (e.getCause().getCause() instanceof javax.jdo.JDOException ||
>               e.getCause().getCause() instanceof NucleusException) {
>             // The JDOException or the Nucleus Exception may be wrapped further in a
MetaException
>             caughtException = e.getCause().getCause();
>        }
> {noformat}
> While in ObjectStore, many places we are only throwing new MetaException(msg) without
the cause, so we are missing retrying for some cases. e.g., with the following JDOException,
we should retry but it's ignored.
> {noformat}
> 2017-04-04 17:28:21,602 ERROR metastore.ObjectStore (ObjectStore.java:getMTableColumnStatistics(6555))
- Error retrieving statistics via jdo
> javax.jdo.JDOException: Exception thrown when executing query
>         at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:596)
>         at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:321)
>         at org.apache.hadoop.hive.metastore.ObjectStore.getMTableColumnStatistics(ObjectStore.java:6546)
>         at org.apache.hadoop.hive.metastore.ObjectStore.access$1200(ObjectStore.java:171)
>         at org.apache.hadoop.hive.metastore.ObjectStore$9.getJdoResult(ObjectStore.java:6606)
>         at org.apache.hadoop.hive.metastore.ObjectStore$9.getJdoResult(ObjectStore.java:6595)
>         at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2633)
>         at org.apache.hadoop.hive.metastore.ObjectStore.getTableColumnStatisticsInternal(ObjectStore.java:6594)
>         at org.apache.hadoop.hive.metastore.ObjectStore.getTableColumnStatistics(ObjectStore.java:6588)
>         at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:103)
>         at com.sun.proxy.$Proxy0.getTableColumnStatistics(Unknown Source)
>         at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterTableUpdateTableColumnStats(HiveAlterHandler.java:787)
>         at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterTable(HiveAlterHandler.java:247)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_core(HiveMetaStore.java:3809)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_with_environment_context(HiveMetaStore.java:3779)
>         at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
>         at com.sun.proxy.$Proxy3.alter_table_with_environment_context(Unknown Source)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_table_with_environment_context.getResult(ThriftHiveMetastore.java:9617)
>         at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$alter_table_with_environment_context.getResult(ThriftHiveMetastore.java:9601)
>         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>         at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
>         at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
>         at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
>         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message