hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Krisztian Kasa (Jira)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-23406) SharedWorkOptimizer should check nullSortOrders when comparing ReduceSink operators
Date Thu, 07 May 2020 19:06:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-23406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Krisztian Kasa updated HIVE-23406:
----------------------------------
    Description: 
SharedWorkOptimizer does not checks null sort order in ReduceSinkDesc when compares ReduceSink
operators:
 [https://github.com/apache/hive/blob/ca9aba606c4d09b91ee28bf9ee1ae918db8cdfb9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1444]
{code:java}
      ReduceSinkDesc op1Conf = ((ReduceSinkOperator) op1).getConf();
      ReduceSinkDesc op2Conf = ((ReduceSinkOperator) op2).getConf();

      if (StringUtils.equals(op1Conf.getKeyColString(), op2Conf.getKeyColString()) &&
        StringUtils.equals(op1Conf.getValueColsString(), op2Conf.getValueColsString()) &&
        StringUtils.equals(op1Conf.getParitionColsString(), op2Conf.getParitionColsString())
&&
        op1Conf.getTag() == op2Conf.getTag() &&
        StringUtils.equals(op1Conf.getOrder(), op2Conf.getOrder()) &&
        op1Conf.getTopN() == op2Conf.getTopN() &&
        canDeduplicateReduceTraits(op1Conf, op2Conf)) {
        return true;
      } else {
        return false;
      }
{code}
An expression like
{code:java}
        StringUtils.equals(op1Conf.getNullOrder(), op2Conf.getNullOrder()) &&
{code}
should be added.

 

  was:
SharedWorkOptimizer does not checks null sort order in ReduceSinkDesc when compares to ReduceSink
operators:
 [https://github.com/apache/hive/blob/ca9aba606c4d09b91ee28bf9ee1ae918db8cdfb9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1444]
{code:java}
      ReduceSinkDesc op1Conf = ((ReduceSinkOperator) op1).getConf();
      ReduceSinkDesc op2Conf = ((ReduceSinkOperator) op2).getConf();

      if (StringUtils.equals(op1Conf.getKeyColString(), op2Conf.getKeyColString()) &&
        StringUtils.equals(op1Conf.getValueColsString(), op2Conf.getValueColsString()) &&
        StringUtils.equals(op1Conf.getParitionColsString(), op2Conf.getParitionColsString())
&&
        op1Conf.getTag() == op2Conf.getTag() &&
        StringUtils.equals(op1Conf.getOrder(), op2Conf.getOrder()) &&
        op1Conf.getTopN() == op2Conf.getTopN() &&
        canDeduplicateReduceTraits(op1Conf, op2Conf)) {
        return true;
      } else {
        return false;
      }
{code}
An expression like
{code:java}
        StringUtils.equals(op1Conf.getNullOrder(), op2Conf.getNullOrder()) &&
{code}
should be added.

 


> SharedWorkOptimizer should check nullSortOrders when comparing ReduceSink operators
> -----------------------------------------------------------------------------------
>
>                 Key: HIVE-23406
>                 URL: https://issues.apache.org/jira/browse/HIVE-23406
>             Project: Hive
>          Issue Type: Bug
>          Components: Physical Optimizer
>            Reporter: Krisztian Kasa
>            Priority: Major
>
> SharedWorkOptimizer does not checks null sort order in ReduceSinkDesc when compares ReduceSink
operators:
>  [https://github.com/apache/hive/blob/ca9aba606c4d09b91ee28bf9ee1ae918db8cdfb9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1444]
> {code:java}
>       ReduceSinkDesc op1Conf = ((ReduceSinkOperator) op1).getConf();
>       ReduceSinkDesc op2Conf = ((ReduceSinkOperator) op2).getConf();
>       if (StringUtils.equals(op1Conf.getKeyColString(), op2Conf.getKeyColString()) &&
>         StringUtils.equals(op1Conf.getValueColsString(), op2Conf.getValueColsString())
&&
>         StringUtils.equals(op1Conf.getParitionColsString(), op2Conf.getParitionColsString())
&&
>         op1Conf.getTag() == op2Conf.getTag() &&
>         StringUtils.equals(op1Conf.getOrder(), op2Conf.getOrder()) &&
>         op1Conf.getTopN() == op2Conf.getTopN() &&
>         canDeduplicateReduceTraits(op1Conf, op2Conf)) {
>         return true;
>       } else {
>         return false;
>       }
> {code}
> An expression like
> {code:java}
>         StringUtils.equals(op1Conf.getNullOrder(), op2Conf.getNullOrder()) &&
> {code}
> should be added.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message