hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gunther Hagleitner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4179) NonBlockingOpDeDup does not merge SEL operators correctly
Date Fri, 15 Mar 2013 23:20:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603970#comment-13603970
] 

Gunther Hagleitner commented on HIVE-4179:
------------------------------------------

Query:

{noformat}
insert overwrite table outputTbl1
SELECT a.key, concat(a.values, a.values), concat(a.values, a.values)
FROM (
  SELECT key, count(1) as values from inputTbl1 group by key
  UNION ALL
  SELECT key, count(1) as values from inputTbl1 group by key
) a;
{noformat}

Before:

{noformat}
          outputColumnNames: _col0, _col1
          Select Operator
            expressions:
                  expr: _col0
                  type: string
                  expr: UDFToLong(_col1)
                  type: bigint
                  expr: UDFToLong(_col2)
                  type: bigint
            outputColumnNames: _col0, _col1, _col2
{noformat}

After:

{noformat}        
          outputColumnNames: _col0, _col1
          Select Operator
            expressions:
                  expr: _col0
                  type: string
                  expr: UDFToLong(concat(_col1, _col1))
                  type: bigint
                  expr: UDFToLong(concat(_col1, _col1))
                  type: bigint
            outputColumnNames: _col0, _col1, _col2
{noformat}
                
> NonBlockingOpDeDup does not merge SEL operators correctly
> ---------------------------------------------------------
>
>                 Key: HIVE-4179
>                 URL: https://issues.apache.org/jira/browse/HIVE-4179
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gunther Hagleitner
>            Assignee: Gunther Hagleitner
>         Attachments: HIVE-4179.1.patch, HIVE-4179.2.patch
>
>
> The input columns list for SEL operations isn't merged properly in the optimization.
The best way to see this is running union_remove_22.q with -Dhadoop.mr.rev=23. The plan shows
lost UDFs and a broken lineage for one column.
> Note: union_remove tests do not run on hadoop 1 or 0.20.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message