hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt McCline (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7426) ClassCastException: ...IntWritable cannot be cast to ...Text involving ql.udf.generic.GenericUDFBasePad.evaluate
Date Mon, 04 Aug 2014 21:38:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14085334#comment-14085334
] 

Matt McCline commented on HIVE-7426:
------------------------------------

From: Jason Dere
12:52 AM (13 hours ago)

Looking at the stack trace and code, GenericUDFBasePad (line 65) expects the 3 UDF arguments
to be of type Text, IntWritable, Text, respectively.  However, the argument checking done
in initialize() only enforces that all 3 arg types are text/int, but it doesn't check that
arg 1 is text, arg 2 is int, arg3 is text. So it looks like the query passed in an int value
to arg1, and the converter returned an IntWritable where it expected a Text. The generic checkArguments()
didn't do its job here - probably needed to have an additional param specifying what the expected
type should be.

What it probably should have done is allow arg1/arg3 to be almost any type, but to convert
the value to string/text during evaluate().  That would allow the int passed in arg1 to be
implicitly cast to string, and the query as written would work.  Or if we want to be strict
with the types, Hive could actually throw an error when checking the param types in initialize().
 In this case then the user would have to explicitly cast the integer value to string such
as LPAD(CAST(<integer_expression> AS STRING), 2, 0).

> ClassCastException: ...IntWritable cannot be cast to ...Text involving ql.udf.generic.GenericUDFBasePad.evaluate
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-7426
>                 URL: https://issues.apache.org/jira/browse/HIVE-7426
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Matt McCline
>         Attachments: TestWithORC.zip, fail_366.sql, fail_750.sql, fail_856.sql
>
>
> One of several found by Raj Bains.
> M/R or Tez.
> Query does not vectorize, so this is not vector related.
> Query:
> {code}
> SELECT `Calcs`.`datetime0` AS `none_datetime0_ok`,   `Calcs`.`int1` AS `none_int1_ok`,
  `Calcs`.`key` AS `none_key_nk`,   CASE WHEN (`Calcs`.`datetime0` IS NOT NULL AND `Calcs`.`int1`
IS NOT NULL) THEN FROM_UNIXTIME(UNIX_TIMESTAMP(CONCAT((YEAR(`Calcs`.`datetime0`)+FLOOR((MONTH(`Calcs`.`datetime0`)+`Calcs`.`int1`)/12)),
CONCAT('-', CONCAT(LPAD(PMOD(MONTH(`Calcs`.`datetime0`)+`Calcs`.`int1`, 12), 2, '0'), SUBSTR(`Calcs`.`datetime0`,
8)))), SUBSTR('yyyy-MM-dd HH:mm:ss',0,LENGTH(`Calcs`.`datetime0`))), 'yyyy-MM-dd HH:mm:ss')
END AS `none_z_dateadd_month_ok` FROM `default`.`testv1_Calcs` `Calcs` GROUP BY `Calcs`.`datetime0`,
  `Calcs`.`int1`,   `Calcs`.`key`,   CASE WHEN (`Calcs`.`datetime0` IS NOT NULL AND `Calcs`.`int1`
IS NOT NULL) THEN FROM_UNIXTIME(UNIX_TIMESTAMP(CONCAT((YEAR(`Calcs`.`datetime0`)+FLOOR((MONTH(`Calcs`.`datetime0`)+`Calcs`.`int1`)/12)),
CONCAT('-', CONCAT(LPAD(PMOD(MONTH(`Calcs`.`datetime0`)+`Calcs`.`int1`, 12), 2, '0'), SUBSTR(`Calcs`.`datetime0`,
8)))), SUBSTR('yyyy-MM-dd HH:mm:ss',0,LENGTH(`Calcs`.`datetime0`))), 'yyyy-MM-dd HH:mm:ss')
END ;
> {code}
> Stack Trace:
> {code}
> Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to org.apache.hadoop.io.Text
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBasePad.evaluate(GenericUDFBasePad.java:65)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.stringEvaluate(GenericUDFConcat.java:189)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.evaluate(GenericUDFConcat.java:159)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.stringEvaluate(GenericUDFConcat.java:189)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.evaluate(GenericUDFConcat.java:159)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.stringEvaluate(GenericUDFConcat.java:189)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFConcat.evaluate(GenericUDFConcat.java:159)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp.evaluate(GenericUDFToUnixTimeStamp.java:121)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFUnixTimeStamp.evaluate(GenericUDFUnixTimeStamp.java:52)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:177)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.udf.generic.GenericUDFWhen.evaluate(GenericUDFWhen.java:78)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
> 	at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
> 	at org.apache.hadoop.hive.ql.exec.KeyWrapperFactory$ListKeyWrapper.getNewKey(KeyWrapperFactory.java:113)
> 	at org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:778)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message