hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-384) problem in union if the first subquery is a map-only job
Date Mon, 06 Apr 2009 16:56:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12696173#action_12696173
] 

Namit Jain commented on HIVE-384:
---------------------------------

The patch has a few bug fixes:

1. The total number of jobs were counted wrongly, which made it very difficult to debug (specially
big union queries).
2. The job name for a query was incorrect, , which made it very difficult to debug (specially
big union queries).

These two are fixed in Driver.java

3. Union plans were very complex and inefficient - since the unions were not getting merged
- the simple fix to to merge 
them in the SemanticAnalyzer - no need to check anything, union schema should be the same.

4. No special case for ReduceSink followed by Union -- the fix is in GenMRRedSink3


Added a bunch of new tests




> problem in union if the first subquery is a map-only job
> --------------------------------------------------------
>
>                 Key: HIVE-384
>                 URL: https://issues.apache.org/jira/browse/HIVE-384
>             Project: Hadoop Hive
>          Issue Type: Bug
>            Reporter: Namit Jain
>            Assignee: Namit Jain
>         Attachments: hive.384.1.patch, hive.384.2.patch, hive.384.3.patch, hive.384.4.patch
>
>
> Union needs special handling.
> explain 
>   select unionsrc.key, count(1) FROM (select s2.key as key, s2.value as value from src1
s2
>                                         UNION  ALL  
>                                       select 'tst1' as key, cast(count(1) as string)
as value from src s1) 
>   unionsrc group by unionsrc.key;
> results in a null pointer exception

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message