hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Navis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3926) PPD on virtual column of partitioned table is not working
Date Sun, 11 Aug 2013 03:46:47 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736175#comment-13736175
] 

Navis commented on HIVE-3926:
-----------------------------

[~sershe] Compacting pruner predicate only removes nulls. It does not do removing non-partition
column related expression. Without this, exception is thrown in the pruneBySequentialScan()
method, like this,
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: cannot
find field block__offset__inside__file from [org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@12fb063,
org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@1e55d39,
org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@14b525c,
org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@c4c05]
	at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.prune(PartitionPruner.java:231)
	at org.apache.hadoop.hive.ql.optimizer.pcr.PcrOpProcFactory$FilterPCR.process(PcrOpProcFactory.java:112)
	... 26 more
Caused by: java.lang.RuntimeException: cannot find field block__offset__inside__file from
[org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@12fb063,
org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@1e55d39,
org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@14b525c,
org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector$MyField@c4c05]
	at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:368)
	at org.apache.hadoop.hive.serde2.objectinspector.UnionStructObjectInspector.getStructFieldRef(UnionStructObjectInspector.java:100)
	at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:55)
	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:121)
	at org.apache.hadoop.hive.ql.optimizer.ppr.PartExprEvalUtils.prepareExpr(PartExprEvalUtils.java:100)
	at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.pruneBySequentialScan(PartitionPruner.java:330)
	at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.prune(PartitionPruner.java:219)
	... 27 more
{noformat}
                
> PPD on virtual column of partitioned table is not working
> ---------------------------------------------------------
>
>                 Key: HIVE-3926
>                 URL: https://issues.apache.org/jira/browse/HIVE-3926
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Navis
>            Assignee: Navis
>            Priority: Minor
>             Fix For: 0.12.0
>
>         Attachments: HIVE-3926.6.patch, HIVE-3926.D8121.1.patch, HIVE-3926.D8121.2.patch,
HIVE-3926.D8121.3.patch, HIVE-3926.D8121.4.patch, HIVE-3926.D8121.5.patch
>
>
> {code}
> select * from src where BLOCK__OFFSET__INSIDE__FILE<100;
> {code}
> is working, but
> {code}
> select * from srcpart where BLOCK__OFFSET__INSIDE__FILE<100;
> {code}
> throws SemanticException. Disabling PPD makes it work.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message