hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-23730) Compiler support tracking TS keyColName for Probe MapJoin
Date Mon, 22 Jun 2020 11:06:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-23730?focusedWorklogId=449159&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-449159
]

ASF GitHub Bot logged work on HIVE-23730:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 22/Jun/20 11:05
            Start Date: 22/Jun/20 11:05
    Worklog Time Spent: 10m 
      Work Description: pgaref commented on a change in pull request #1152:
URL: https://github.com/apache/hive/pull/1152#discussion_r443482036



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
##########
@@ -1566,13 +1569,38 @@ private void removeSemijoinsParallelToMapJoin(OptimizeTezProcContext
procCtx)
 
       List<ExprNodeDesc> keyDesc = selectedMJOp.getConf().getKeys().get(posBigTable);
       ExprNodeColumnDesc keyCol = (ExprNodeColumnDesc) keyDesc.get(0);
-
-      tsProbeDecodeCtx = new TableScanOperator.ProbeDecodeContext(mjCacheKey, mjSmallTablePos,
-          keyCol.getColumn(), selectedMJOpRatio);
+      String realTSColName = getOriginalTSColName(selectedMJOp, keyCol.getColumn());
+      if (realTSColName != null) {
+        tsProbeDecodeCtx = new TableScanOperator.ProbeDecodeContext(mjCacheKey, mjSmallTablePos,
+                realTSColName, selectedMJOpRatio);
+      } else {
+        LOG.warn("ProbeDecode could not find TSColName for ColKey {} with MJ Schema {} ",
keyCol, selectedMJOp.getSchema());

Review comment:
       Hey @jcamachor , thanks for the comments!
   The HIVE_IN_TEST trick could work as long we enable probedecode optimisation by default
right? (currently if off)
   
   Just enabled the optimisation for this PR (throwing an exception instead of warn) to identify
any existing issues.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 449159)
    Time Spent: 40m  (was: 0.5h)

> Compiler support tracking TS keyColName for Probe MapJoin
> ---------------------------------------------------------
>
>                 Key: HIVE-23730
>                 URL: https://issues.apache.org/jira/browse/HIVE-23730
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Panagiotis Garefalakis
>            Assignee: Panagiotis Garefalakis
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> Compiler needs to track the original TS key columnName used for MJ probedecode.
> Even though we know the MJ keyCol at compile time, this could be generated by previous
(parent) operators thus we dont always know the original TS column it maps to.
> To find the original columnMapping, we need to track the MJ keyCol through the operator
pipeline. Tracking can be done through the parent operator ColumnExprMap and RowSchema.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message