hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15338) Wrong result from non-vectorized DATEDIFF with scalar parameter of type DATE/TIMESTAMP
Date Mon, 12 Dec 2016 07:01:08 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15741163#comment-15741163
] 

Matt McCline commented on HIVE-15338:
-------------------------------------


The change making the date constant a Long instead of an Integer causes this stack trace for
vector_between_in.q --
{code}
2016-12-11T22:53:13,060 DEBUG [614501c2-eb7b-432f-b990-6bcd8abe7e5d main] physical.Vectorizer:
Failed to vectorize
java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer
	at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getInExpression(VectorizationContext.java:1712)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getGenericUdfVectorExpression(VectorizationContext.java:1335)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:541)
~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateExprNodeDesc(Vectorizer.java:1878)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateFilterOperator(Vectorizer.java:1604)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateMapWorkOperator(Vectorizer.java:1424)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$MapWorkValidationNodeProcessor.process(Vectorizer.java:1081)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:158) [hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.validateMapWork(Vectorizer.java:886)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.convertMapWork(Vectorizer.java:479)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer$VectorizationDispatcher.dispatch(Vectorizer.java:453)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.dispatch(TaskGraphWalker.java:111) [hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.walk(TaskGraphWalker.java:180) [hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.lib.TaskGraphWalker.startWalking(TaskGraphWalker.java:125) [hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.resolve(Vectorizer.java:1406)
[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
{code}

> Wrong result from non-vectorized DATEDIFF with scalar parameter of type DATE/TIMESTAMP
> --------------------------------------------------------------------------------------
>
>                 Key: HIVE-15338
>                 URL: https://issues.apache.org/jira/browse/HIVE-15338
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Matt McCline
>            Assignee: Matt McCline
>            Priority: Critical
>         Attachments: HIVE-15338.01.patch, HIVE-15338.02.patch, HIVE-15338.03.patch, HIVE-15338.04.patch
>
>
> Vectorization in vectorized DATEDIFF accidentally treated the scalar parameter is type
DATE (e.g. CURRENT_DATE) as 0.
> Current Q file test vectorized_date_funcs.q DOES NOT test the DATE/TIMESTAMP scalar type
case.
> And, non-vectorized cases of DATEDIFF are using UTF and returning the wrong results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message