hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vikram Dixit K (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-10907) Hive on Tez: Classcast exception in some cases with SMB joins
Date Sat, 06 Jun 2015 17:43:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14575843#comment-14575843
] 

Vikram Dixit K edited comment on HIVE-10907 at 6/6/15 5:42 PM:
---------------------------------------------------------------

[~sershe] Can you please review this? The purpose of the patch is to prevent smb joins in
cases where one of the sides would be a map side work and the other is coming from a shuffle.
This jira is a work around to stop class cast exceptions from occurring in that case. The
way to prevent this is in ConvertJoinMapJoin code where a check is made to see if the number
of reduce sinks above the parent of the join operator is either 0 or non-zero on all sides
of the join.

a join b

Non-Kosher case:
{noformat}
              | There should be either no RS left of this boundary or there should be one
or more for both sides. If that is not the case, no SMB.
RS -> Gby ->  | RS -> Join ->
TS -> Fil ->  |   RS /
{noformat}

[~hagleitn] reviewed the patch earlier and made a comment that I addressed (basically uncommented
the code). Can you take a look and review this patch please? This needs to go to branch-1.2
as well.

Thanks
Vikram.


was (Author: vikram.dixit):
[~sershe] Can you please review this? The purpose of the patch is to prevent smb joins in
cases where one of the sides would be a map side work and the other is coming from a shuffle.
This jira is a work around to stop class cast exceptions from occurring in that case. The
way to prevent this is in ConvertJoinMapJoin code where a check is made to see if the number
of reduce sinks above the parent of the join operator is either 0 or non-zero on all sides
of the join.

a join b

Non-Kosher case:
{noformat}
                 | There should be either no RS left of this boundary or there should be one
or more for both sides. If that is not the case, no SMB.
RS -> Gby ->  | RS -> Join ->
TS -> Fil -> |   RS /
{noformat}

[~hagleitn] reviewed the patch earlier and made a comment that I addressed (basically uncommented
the code). Can you take a look and review this patch please? This needs to go to branch-1.2
as well.

Thanks
Vikram.

> Hive on Tez: Classcast exception in some cases with SMB joins
> -------------------------------------------------------------
>
>                 Key: HIVE-10907
>                 URL: https://issues.apache.org/jira/browse/HIVE-10907
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>         Attachments: HIVE-10907.1.patch, HIVE-10907.2.patch, HIVE-10907.3.patch, HIVE-10907.4.patch
>
>
> In cases where there is a mix of Map side work and reduce side work, we get a classcast
exception because we assume homogeneity in the code. We need to fix this correctly. For now
this is a workaround.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message