hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lefty Leverenz (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-10690) ArrayIndexOutOfBounds exception in MetaStoreDirectSql.aggrColStatsForPartitions()
Date Sat, 23 May 2015 11:24:17 GMT

     [ https://issues.apache.org/jira/browse/HIVE-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lefty Leverenz updated HIVE-10690:
----------------------------------
    Fix Version/s: 1.2.1

> ArrayIndexOutOfBounds exception in MetaStoreDirectSql.aggrColStatsForPartitions()
> ---------------------------------------------------------------------------------
>
>                 Key: HIVE-10690
>                 URL: https://issues.apache.org/jira/browse/HIVE-10690
>             Project: Hive
>          Issue Type: Bug
>          Components: Statistics
>    Affects Versions: 1.2.0
>            Reporter: Jason Dere
>            Assignee: Vaibhav Gumashta
>             Fix For: 1.3.0, 1.2.1
>
>         Attachments: HIVE-10690.1.patch
>
>
> Noticed a bunch of these stack traces in hive.log while running some unit tests:
> {noformat}
> 2015-05-11 21:18:59,371 WARN  [main]: metastore.ObjectStore (ObjectStore.java:handleDirectSqlError(2420))
- Direct SQL failed
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>         at java.util.ArrayList.rangeCheck(ArrayList.java:635)
>         at java.util.ArrayList.get(ArrayList.java:411)
>         at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.aggrColStatsForPartitions(MetaStoreDirectSql.java:1132)
>         at org.apache.hadoop.hive.metastore.ObjectStore$8.getSqlResult(ObjectStore.java:6162)
>         at org.apache.hadoop.hive.metastore.ObjectStore$8.getSqlResult(ObjectStore.java:6158)
>         at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:2385)
>         at org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:6158)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
>         at com.sun.proxy.$Proxy84.get_aggr_stats_for(Unknown Source)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_aggr_stats_for(HiveMetaStore.java:5662)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
>         at com.sun.proxy.$Proxy86.get_aggr_stats_for(Unknown Source)
>         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAggrColStatsFor(HiveMetaStoreClient.java:2064)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156)
>         at com.sun.proxy.$Proxy87.getAggrColStatsFor(Unknown Source)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getAggrColStatsFor(Hive.java:3110)
>         at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:245)
>         at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.updateColStats(RelOptHiveTable.java:329)
>         at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getColStat(RelOptHiveTable.java:399)
>         at org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable.getColStat(RelOptHiveTable.java:392)
>         at org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan.getColStat(HiveTableScan.java:150)
>         at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdDistinctRowCount.getDistinctRowCount(HiveRelMdDistinctRowCount.java:77)
>         at org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdDistinctRowCount.getDistinctRowCount(HiveRelMdDistinctRowCount.java:64)
>         at sun.reflect.GeneratedMethodAccessor296.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke(ReflectiveRelMetadataProvider.java:182)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:132)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:132)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:469)
>         at org.apache.calcite.rel.metadata.RelMdDistinctRowCount.getDistinctRowCount(RelMdDistinctRowCount.java:241)
>         at sun.reflect.GeneratedMethodAccessor300.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke(ReflectiveRelMetadataProvider.java:182)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:132)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke(ChainedRelMetadataProvider.java:109)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.calcite.rel.metadata.CachingRelMetadataProvider$CachingInvocationHandler.invoke(CachingRelMetadataProvider.java:132)
>         at com.sun.proxy.$Proxy108.getDistinctRowCount(Unknown Source)
>         at org.apache.calcite.rel.metadata.RelMetadataQuery.getDistinctRowCount(RelMetadataQuery.java:469)
>         at org.apache.calcite.rel.rules.LoptOptimizeJoinRule.computeJoinCardinality(LoptOptimizeJoinRule.java:615)
>         at org.apache.calcite.rel.rules.LoptOptimizeJoinRule.getBestNextFactor(LoptOptimizeJoinRule.java:808)
>         at org.apache.calcite.rel.rules.LoptOptimizeJoinRule.createOrdering(LoptOptimizeJoinRule.java:706)
>         at org.apache.calcite.rel.rules.LoptOptimizeJoinRule.findBestOrderings(LoptOptimizeJoinRule.java:458)
>         at org.apache.calcite.rel.rules.LoptOptimizeJoinRule.onMatch(LoptOptimizeJoinRule.java:128)
>         at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:326)
>         at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:515)
>         at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:392)
>         at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:255)
>         at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:125)
>         at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:207)
>         at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:194)
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:856)
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:768)
>         at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:109)
>         at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:730)
>         at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:145)
>         at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:105)
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:607)
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:244)
>         at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10051)
>         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:207)
>         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
>         at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
>         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
>         at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
>         at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
>         at org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1029)
>         at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1003)
>         at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.runTest(TestMiniTezCliDriver.java:1234)
>         at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_auto_sortmerge_join_1(TestMiniTezCliDriver.java:166)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at junit.framework.TestCase.runTest(TestCase.java:176)
>         at junit.framework.TestCase.runBare(TestCase.java:141)
>         at junit.framework.TestResult$1.protect(TestResult.java:122)
>         at junit.framework.TestResult.runProtected(TestResult.java:142)
>         at junit.framework.TestResult.run(TestResult.java:125)
>         at junit.framework.TestCase.run(TestCase.java:129)
>         at junit.framework.TestSuite.runTest(TestSuite.java:255)
>         at junit.framework.TestSuite.run(TestSuite.java:250)
>         at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
>         at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
>         at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
>         at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message