hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <>
Subject [jira] [Commented] (HIVE-4502) NPE - subquery smb joins fails
Date Tue, 16 Jul 2013 01:52:49 GMT


Phabricator commented on HIVE-4502:

ashutoshc has requested changes to the revision "HIVE-4502 [jira] NPE - subquery smb joins

  I dont like the idea of tracking operators in the task thereby tasks getting involved in
plan manipulation but in absence of alternatives, this may be the way to move forward for

  ql/src/java/org/apache/hadoop/hive/ql/ I guess this is unintentional.
  ql/src/java/org/apache/hadoop/hive/ql/exec/ I am little hesistant about task
being aware of operators. Ideally if we have good demarkation of different layers, task (which
is sort of a driver to execute a plan) need not be aware of operators contained in its plan.
Planner shouldn't wait so late that it needs to track operators in task to generate plan correctly.
I don't have a proposal on how to do this better though. At the very least, it will help to
add comment explaining the purpose of this list.
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ I think its better
to do following:
  If (op.getNumChild() != 1){
    throw new IllegalStateException("Expecting operator " + op + "to have one child. Found:
" + op.getNumChild());
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ It will be good
to add comments on what this method intends to do. Also, I guess you can make this protected,
since all the callers of this method are in same package.
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ Does this method
need to be public?
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ I didnt get this
part. Can you add comments why old and curr tasks need to be of type ExecDriver. What about
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ Does this method
need to be public? Also, please add javadocs for the method.
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ throw IllegalStateException
in case # of parents != 1
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ I didnt get why
did you remove this if check. We shall be needing tagging only for joins. Right? Removing
this check implies we will tag always, is that needed?
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ Better name: hasBranchFinished()
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ Why is this function
call needed here? Can you please add comment for it?
  ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out:165 I see that there is no
Stage-2 at all. What caused this? Its little weird that we have stages 0,1 and 3 in plan and
no stage-2 at all.
  ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out:349 No stage-2 here either.
Probably the same cause?




To: JIRA, ashutoshc, navis
Cc: brock

> NPE - subquery smb joins fails
> ------------------------------
>                 Key: HIVE-4502
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 0.11.0
>            Reporter: Vikram Dixit K
>            Assignee: Navis
>         Attachments: HIVE-4502.D10695.1.patch, HIVE-4502.D10695.2.patch, HIVE-4502.D10695.3.patch,
smb_mapjoin_25.q, smb_mapjoin_25.q
> Found this issue while running some SMB joins. Attaching test case that causes this error.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message