hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arvind Prabhakar (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-1271) Case sensitiveness of type information specified when using custom reducer causes type mismatch
Date Sat, 27 Mar 2010 02:40:27 GMT

     [ https://issues.apache.org/jira/browse/HIVE-1271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arvind Prabhakar updated HIVE-1271:
-----------------------------------

    Status: Patch Available  (was: Open)

*Summary*
The implementation of {{equals()}} method of {{StructTypeInfo}} was comparing field names
as part of the comparison. This is not valid since field namess do not contitute the definition
of a type. This patch refactors the {{TypedInfo}} hierarchy to address this issue.
    
*Implementation Details*
- Modified the {{TypedInfo}} and removed its implementation of the {{equals()}} method.
- Modified all specialized subclasses to make them {{final}}.
- Modified all subclass implementation of {{equals()}} to skip category comparison.
- Modified {{StructTypeInfo}} implementation of {{equals()}} to not compare field names.

*Testing Done*
- Built and tested the usecase identified in this issue. It works now.
- Ran full set of tests. Out of these two tests - clientpositive for input20.q and input33.q
failed for unrelated reasons (these tests are failing on the trunk as well). All other tests
passed.

> Case sensitiveness of type information specified when using custom reducer causes type
mismatch
> -----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-1271
>                 URL: https://issues.apache.org/jira/browse/HIVE-1271
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.5.0
>            Reporter: Dilip Joseph
>            Assignee: Arvind Prabhakar
>         Attachments: HIVE-1271.patch
>
>
> Type information specified  while using a custom reduce script is converted to lower
case, and causes type mismatch during query semantic analysis .  The following REDUCE query
where field name =  "userId" failed.
> hive> CREATE TABLE SS (
>    >                     a INT,
>    >                     b INT,
>    >                     vals ARRAY<STRUCT<userId:INT, y:STRING>>
>    >                 );
> OK
> hive> FROM (select * from srcTable DISTRIBUTE BY id SORT BY id) s
>    >     INSERT OVERWRITE TABLE SS
>    >     REDUCE *
>    >         USING 'myreduce.py'
>    >         AS
>    >                     (a INT,
>    >                     b INT,
>    >                     vals ARRAY<STRUCT<userId:INT, y:STRING>>
>    >                     )
>    >         ;
> FAILED: Error in semantic analysis: line 2:27 Cannot insert into
> target table because column number/types are different SS: Cannot
> convert column 2 from array<struct<userId:int,y:string>> to
> array<struct<userid:int,y:string>>.
> The same query worked fine after changing "userId" to "userid".

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message