hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arvind Prabhakar (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1271) Case sensitiveness of type information specified when using custom reducer causes type mismatch
Date Tue, 22 Jun 2010 21:30:55 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881368#action_12881368

Arvind Prabhakar commented on HIVE-1271:

@Ashish: Thanks for looking at the patch. 

bq. why remove the check on Category?

I modified all the specialized type infos to be {{final}} - which in turn ensures that if
the test on {{instanceof}} succeeds, then they have to be the same category type. Therefore,
the check on category was redundant going forward.

bq. Also why drop the default implementation of the equals method for TypeInfo?

I did this for two main reasons - first that fact that it was implementing the {{equals()}}
but not {{hashCode()}} method. This could lead to unexpected behavior when {{TypeInfo}} instances
were put in collections. Second, the implementation was modified to make both {{equals()}}
and {{hashCode()}} methods to be made abstract in order to force any (new) child classes to
make sure that they implement both consistently.

Let me know if you would like to tweak this change as necessary.

> Case sensitiveness of type information specified when using custom reducer causes type
> -----------------------------------------------------------------------------------------------
>                 Key: HIVE-1271
>                 URL: https://issues.apache.org/jira/browse/HIVE-1271
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.5.0
>            Reporter: Dilip Joseph
>            Assignee: Arvind Prabhakar
>             Fix For: 0.6.0
>         Attachments: HIVE-1271-1.patch, HIVE-1271.patch
> Type information specified  while using a custom reduce script is converted to lower
case, and causes type mismatch during query semantic analysis .  The following REDUCE query
where field name =  "userId" failed.
>    >                     a INT,
>    >                     b INT,
>    >                     vals ARRAY<STRUCT<userId:INT, y:STRING>>
>    >                 );
> OK
> hive> FROM (select * from srcTable DISTRIBUTE BY id SORT BY id) s
>    >     REDUCE *
>    >         USING 'myreduce.py'
>    >         AS
>    >                     (a INT,
>    >                     b INT,
>    >                     vals ARRAY<STRUCT<userId:INT, y:STRING>>
>    >                     )
>    >         ;
> FAILED: Error in semantic analysis: line 2:27 Cannot insert into
> target table because column number/types are different SS: Cannot
> convert column 2 from array<struct<userId:int,y:string>> to
> array<struct<userid:int,y:string>>.
> The same query worked fine after changing "userId" to "userid".

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message