hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7421) Null pointer exception involving ql.exec.vector.expressions.StringConcatColScalar.evaluate
Date Wed, 06 Aug 2014 01:50:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087101#comment-14087101
] 

Matt McCline commented on HIVE-7421:
------------------------------------


*Postgres* results (using necessarily modified query):
{code}
mmccline=# SELECT order_priority AS none_order_priority_nk FROM testv1_Staples WHERE ((CONCAT(TO_DATE(order_date_,'YYYY-MM-DD'),
' 00:00:00') = '1997-01-01 00:00:00' OR CONCAT(TO_DATE(order_date_,'YYYY-MM-DD'), ' 00:00:00')
= '1997-01-03 00:00:00') AND (TO_DATE(order_date_,'YYYY-MM-DD') = '1997-01-01' OR TO_DATE(order_date_,'YYYY-MM-DD')
= '1997-01-03')) GROUP BY order_priority;
 none_order_priority_nk 
------------------------
 4-NOT SPECIFIED
 1-URGENT
 2-HIGH
 3-MEDIUM
 5-LOW
(5 rows)
{code}

Non-vectorized, M/R or Tez:
{code}
SELECT `Staples`.`order_priority` AS `none_order_priority_nk` FROM `default`.`testv1_Staples`
`Staples` WHERE ((CONCAT(TO_DATE(`Staples`.`order_date_`), ' 00:00:00') = '1997-01-01 00:00:00'
OR CONCAT(TO_DATE(`Staples`.`order_date_`), ' 00:00:00') = '1997-01-03 00:00:00') AND (TO_DATE(`Staples`.`order_date_`)
= '1997-01-01' OR TO_DATE(`Staples`.`order_date_`) = '1997-01-03')) GROUP BY `Staples`.`order_priority`
;
1-URGENT
2-HIGH
3-MEDIUM
4-NOT SPECIFIED
5-LOW
{code}

*NO RESULTS* when Vectorized.

> Null pointer exception involving ql.exec.vector.expressions.StringConcatColScalar.evaluate
> ------------------------------------------------------------------------------------------
>
>                 Key: HIVE-7421
>                 URL: https://issues.apache.org/jira/browse/HIVE-7421
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Matt McCline
>            Assignee: Matt McCline
>         Attachments: TestWithORC.zip, fail_47.sql, fail_62.sql, fail_932.sql
>
>
> One of several found by Raj Bains.
> M/R or Tez.
> {code}
> set hive.vectorized.execution.enabled=true;
> {code}
> Seems very similar to https://issues.apache.org/jira/browse/HIVE-6649
> Query:
> {code}
> SELECT FLOOR((7 + DATEDIFF(`Staples`.`order_date_`, CONCAT(CAST(YEAR(`Staples`.`order_date_`)
AS STRING), '-01-01 00:00:00'))  +pmod(8 + pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`)
AS STRING), '-01-01 00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) AS `wk_order_date_ok`,  
SUM(`Staples`.`sales_total`) AS `sum_sales_total_ok` FROM `default`.`testv1_Staples` `Staples`
GROUP BY FLOOR((7 + DATEDIFF(`Staples`.`order_date_`, CONCAT(CAST(YEAR(`Staples`.`order_date_`)
AS STRING), '-01-01 00:00:00'))  +pmod(8 + pmod(datediff(CONCAT(CAST(YEAR(`Staples`.`order_date_`)
AS STRING), '-01-01 00:00:00'), '1995-01-01'), 7) - 2, 7) ) / 7) ;
> {code}
> Stack trace:
> {code}
> Caused by: java.lang.NullPointerException
> 	at java.lang.System.arraycopy(Native Method)
> 	at org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setConcat(BytesColumnVector.java:190)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.StringConcatColScalar.evaluate(StringConcatColScalar.java:78)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFDateDiffColCol.evaluate(VectorUDFDateDiffColCol.java:59)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongScalarAddLongColumn.evaluate(LongScalarAddLongColumn.java:65)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.gen.LongColAddLongColumn.evaluate(LongColAddLongColumn.java:52)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.LongColDivideLongScalar.evaluate(LongColDivideLongScalar.java:52)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpression.evaluateChildren(VectorExpression.java:112)
> 	at org.apache.hadoop.hive.ql.exec.vector.expressions.gen.FuncFloorDoubleToLong.evaluate(FuncFloorDoubleToLong.java:47)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.evaluateBatch(VectorHashKeyWrapperBatch.java:147)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.processBatch(VectorGroupByOperator.java:289)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:711)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
> 	at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
> 	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
> 	at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message