hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gunther Hagleitner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15279) map join dummy operators are not set up correctly in certain cases with merge join
Date Mon, 05 Dec 2016 22:33:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15723582#comment-15723582
] 

Gunther Hagleitner commented on HIVE-15279:
-------------------------------------------

Hm. Can you explain how this works now/ why you had to change the patch (equals/hash, etc)?

{noformat}
125      l4j.info("Main work is " + reduceWork.getName());
126	    List<HashTableDummyOperator> workOps = reduceWork.getDummyOps();
127	    HashSet<HashTableDummyOperator> dummyOps = workOps == null ? null : new HashSet<>(workOps);
126	    tagToReducerMap.put(redWork.getTag(), redWork);	128	    tagToReducerMap.put(redWork.getTag(),
redWork);
127	    if (mergeWorkList != null) {	129	    if (mergeWorkList != null) {
128	      for (BaseWork mergeWork : mergeWorkList) {	130	      for (BaseWork mergeWork : mergeWorkList)
{
131	        if (l4j.isDebugEnabled()) {
132	          l4j.debug("Additional work " + mergeWork.getName());
133	        }
134	        workOps = mergeWork.getDummyOps();
135	        if (workOps != null) {
136	          if (dummyOps == null) {
137	            dummyOps = new HashSet<>(workOps);
138	          } else {
139	            dummyOps.addAll(workOps);
140	          }
{noformat}

This seems unnecessarily confusing to me. Can't we just always have empty lists/hash instead
of distinguishing between empty and null?

Also, before commit can you either resolve or remove the TODO? 

{noformat}
LOG.debug("Setting dummy ops for work " + work.getName() + ": " + dummyOps);
{noformat}

Isn't really that helpful to just know the id of the list of dummy ops, is it?


> map join dummy operators are not set up correctly in certain cases with merge join
> ----------------------------------------------------------------------------------
>
>                 Key: HIVE-15279
>                 URL: https://issues.apache.org/jira/browse/HIVE-15279
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-15279.01.patch, HIVE-15279.02.patch, HIVE-15279.patch
>
>
> As a result, MapJoin is not initialized and there's NPE later.
> Tez-specific.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message