hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
Date Mon, 09 Feb 2015 22:12:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312987#comment-14312987
] 

Chao commented on HIVE-9574:
----------------------------

Regarding cbo_gby_empty.q, I found this error in hive.log:

{noformat}
...
NestedThrowablesStackTrace:
Method/Identifier expected at character 37 in "tableName == t1 && dbName == t2 &&
()"
org.datanucleus.store.query.QueryCompilerSyntaxException: Method/Identifier expected at character
37 in "tableName == t1 && dbName == t2 && ()"
	at org.datanucleus.query.compiler.JDOQLParser.processPrimary(JDOQLParser.java:810)
	at org.datanucleus.query.compiler.JDOQLParser.processUnaryExpression(JDOQLParser.java:656)
	at org.datanucleus.query.compiler.JDOQLParser.processMultiplicativeExpression(JDOQLParser.java:582)
	at org.datanucleus.query.compiler.JDOQLParser.processAdditiveExpression(JDOQLParser.java:553)
	at org.datanucleus.query.compiler.JDOQLParser.processRelationalExpression(JDOQLParser.java:467)
	at org.datanucleus.query.compiler.JDOQLParser.processAndExpression(JDOQLParser.java:450)
	at org.datanucleus.query.compiler.JDOQLParser.processExclusiveOrExpression(JDOQLParser.java:436)
	at org.datanucleus.query.compiler.JDOQLParser.processInclusiveOrExpression(JDOQLParser.java:422)
	at org.datanucleus.query.compiler.JDOQLParser.processConditionalAndExpression(JDOQLParser.java:408)
	at org.datanucleus.query.compiler.JDOQLParser.processConditionalOrExpression(JDOQLParser.java:389)
	at org.datanucleus.query.compiler.JDOQLParser.processExpression(JDOQLParser.java:378)
	at org.datanucleus.query.compiler.JDOQLParser.processPrimary(JDOQLParser.java:785)
	at org.datanucleus.query.compiler.JDOQLParser.processUnaryExpression(JDOQLParser.java:656)
	at org.datanucleus.query.compiler.JDOQLParser.processMultiplicativeExpression(JDOQLParser.java:582)
	at org.datanucleus.query.compiler.JDOQLParser.processAdditiveExpression(JDOQLParser.java:553)
	at org.datanucleus.query.compiler.JDOQLParser.processRelationalExpression(JDOQLParser.java:467)
	at org.datanucleus.query.compiler.JDOQLParser.processAndExpression(JDOQLParser.java:450)
	at org.datanucleus.query.compiler.JDOQLParser.processExclusiveOrExpression(JDOQLParser.java:436)
	at org.datanucleus.query.compiler.JDOQLParser.processInclusiveOrExpression(JDOQLParser.java:422)
	at org.datanucleus.query.compiler.JDOQLParser.processConditionalAndExpression(JDOQLParser.java:412)
	at org.datanucleus.query.compiler.JDOQLParser.processConditionalOrExpression(JDOQLParser.java:389)
	at org.datanucleus.query.compiler.JDOQLParser.processExpression(JDOQLParser.java:378)
	at org.datanucleus.query.compiler.JDOQLParser.parse(JDOQLParser.java:99)
	at org.datanucleus.query.compiler.JavaQueryCompiler.compileFilter(JavaQueryCompiler.java:467)
	at org.datanucleus.query.compiler.JDOQLCompiler.compile(JDOQLCompiler.java:113)
	at org.datanucleus.store.query.AbstractJDOQLQuery.compileInternal(AbstractJDOQLQuery.java:367)
	at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:240)
	at org.datanucleus.store.query.Query.executeQuery(Query.java:1744)
	at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672)
	at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:312)
	at org.apache.hadoop.hive.metastore.ObjectStore.getMTableColumnStatistics(ObjectStore.java:5969)
	at org.apache.hadoop.hive.metastore.ObjectStore.access$1000(ObjectStore.java:159)
	at org.apache.hadoop.hive.metastore.ObjectStore$6.getJdoResult(ObjectStore.java:6025)
	at org.apache.hadoop.hive.metastore.ObjectStore$6.getJdoResult(ObjectStore.java:6017)
{noformat}

I'm not sure what this mean.

> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
> --------------------------------------------------------------------------------
>
>                 Key: HIVE-9574
>                 URL: https://issues.apache.org/jira/browse/HIVE-9574
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Rui Li
>            Assignee: Jimmy Xiang
>             Fix For: spark-branch
>
>         Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch, HIVE-9574.3-spark.patch,
HIVE-9574.4-spark.patch, HIVE-9574.5-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is expensive. If we
switch {{container}} and {{backupContainer}} frequently in {{HiveKVResultCache}}, it will
downgrade performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message