hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4502) NPE - subquery smb joins fails
Date Tue, 16 Jul 2013 01:52:49 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13709352#comment-13709352
] 

Phabricator commented on HIVE-4502:
-----------------------------------

ashutoshc has requested changes to the revision "HIVE-4502 [jira] NPE - subquery smb joins
fails".

  I dont like the idea of tracking operators in the task thereby tasks getting involved in
plan manipulation but in absence of alternatives, this may be the way to move forward for
now.

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java:619 I guess this is unintentional.
  ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java:72 I am little hesistant about task
being aware of operators. Ideally if we have good demarkation of different layers, task (which
is sort of a driver to execute a plan) need not be aware of operators contained in its plan.
Planner shouldn't wait so late that it needs to track operators in task to generate plan correctly.
I don't have a proposal on how to do this better though. At the very least, it will help to
add comment explaining the purpose of this list.
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRRedSink1.java:70 I think its better
to do following:
  If (op.getNumChild() != 1){
    throw new IllegalStateException("Expecting operator " + op + "to have one child. Found:
" + op.getNumChild());
  }
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:261 It will be good
to add comments on what this method intends to do. Also, I guess you can make this protected,
since all the callers of this method are in same package.
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:281 Does this method
need to be public?
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:310 I didnt get this
part. Can you add comments why old and curr tasks need to be of type ExecDriver. What about
MapredLocalWork?
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:318 Does this method
need to be public? Also, please add javadocs for the method.
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:851 throw IllegalStateException
in case # of parents != 1
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:949 I didnt get why
did you remove this if check. We shall be needing tagging only for joins. Right? Removing
this check implies we will tag always, is that needed?
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java:942 Better name: hasBranchFinished()
?
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinFactory.java:228 Why is this function
call needed here? Can you please add comment for it?
  ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out:165 I see that there is no
Stage-2 at all. What caused this? Its little weird that we have stages 0,1 and 3 in plan and
no stage-2 at all.
  ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out:349 No stage-2 here either.
Probably the same cause?

REVISION DETAIL
  https://reviews.facebook.net/D10695

BRANCH
  HIVE-4502

ARCANIST PROJECT
  hive

To: JIRA, ashutoshc, navis
Cc: brock

                
> NPE - subquery smb joins fails
> ------------------------------
>
>                 Key: HIVE-4502
>                 URL: https://issues.apache.org/jira/browse/HIVE-4502
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.11.0
>            Reporter: Vikram Dixit K
>            Assignee: Navis
>         Attachments: HIVE-4502.D10695.1.patch, HIVE-4502.D10695.2.patch, HIVE-4502.D10695.3.patch,
smb_mapjoin_25.q, smb_mapjoin_25.q
>
>
> Found this issue while running some SMB joins. Attaching test case that causes this error.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message