hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10231) Compute partition column stats fails if partition col type is date
Date Tue, 07 Apr 2015 00:20:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14482271#comment-14482271
] 

Ashutosh Chauhan commented on HIVE-10231:
-----------------------------------------

[~ctang.ma] Since Hive's type system is lucy-gucy you might be able to get away with quoting
all the time, but I won't recommend it. My suggestion will be to rather go on other extreme
and help parser as much as we can about types. So, generate constants in query like following:
{code}
switch (colType):
  case Long:
     colVal+"L";
  case SmallInt:
     colVal+"S";
  case Tinyint:
     colVal+"Y";
  case Decimal:
    colVal+"BD";
  case String:
  case Char:
  case VarChar:
   " ' "+colVal+" ' ";
 case Date:
  date " ' "+colVal+" ' ";
case TimeStamp:
  timestamp " ' "+colVal+" ' "; 
{code} 

> Compute partition column stats fails if partition col type is date
> ------------------------------------------------------------------
>
>                 Key: HIVE-10231
>                 URL: https://issues.apache.org/jira/browse/HIVE-10231
>             Project: Hive
>          Issue Type: Bug
>          Components: Statistics
>    Affects Versions: 1.0.0
>            Reporter: Chaoyu Tang
>            Assignee: Chaoyu Tang
>             Fix For: 1.2.0
>
>         Attachments: HIVE-10231.patch
>
>
> Currently the command "analyze table .. partition .. compute statistics for columns"
may only work for partition column type of string, numeric types, but not others like date.
See following case using date as partition coltype:
> {code}
> create table colstatspartdate (key int, value string) partitioned by (ds date, hr int);
> insert into colstatspartdate partition (ds=date '2015-04-02', hr=2) select key, value
from src limit 20;
> analyze table colstatspartdate partition (ds=date '2015-04-02', hr=2) compute statistics
for columns;
> {code}
> you will get RuntimeException:
> {code}
> FAILED: RuntimeException Cannot convert to Date from: int
> 15/04/06 17:30:01 ERROR ql.Driver: FAILED: RuntimeException Cannot convert to Date from:
int
> java.lang.RuntimeException: Cannot convert to Date from: int
> 	at org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorUtils.getDate(PrimitiveObjectInspectorUtils.java:1048)
> 	at org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$DateConverter.convert(PrimitiveObjectInspectorConverter.java:264)
> 	at org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.typeCast(ConstantPropagateProcFactory.java:163)
> 	at org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.propagate(ConstantPropagateProcFactory.java:333)
> 	at org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:242)
> ....
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message