drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3623) Hive query hangs with limit 0 clause
Date Thu, 19 Nov 2015 22:12:11 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014506#comment-15014506
] 

ASF GitHub Bot commented on DRILL-3623:
---------------------------------------

Github user sudheeshkatkam commented on a diff in the pull request:

    https://github.com/apache/drill/pull/193#discussion_r45408655
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/FindLimit0Visitor.java
---
    @@ -36,16 +48,62 @@
      * executing a schema-only query.
      */
     public class FindLimit0Visitor extends RelShuttleImpl {
    -  private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(FindLimit0Visitor.class);
    +//  private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(FindLimit0Visitor.class);
    +
    +  /**
    +   * Values in the {@link DrillConstExecutor#DRILL_TO_CALCITE_TYPE_MAPPING} map.
    +   * + without {@link SqlTypeName#ANY} (avoid late binding)
    +   * + without {@link SqlTypeName#VARBINARY} ({@link DrillValuesRel values} does not
support this)
    +   * + with {@link SqlTypeName#CHAR} ({@link DrillValuesRel values} supports this, but
the above mapping does not
    +   *   contain this type.
    +   */
    +  private static final ImmutableSet<SqlTypeName> TYPES = ImmutableSet.<SqlTypeName>builder()
    +      .add(SqlTypeName.INTEGER, SqlTypeName.BIGINT, SqlTypeName.FLOAT, SqlTypeName.DOUBLE,
SqlTypeName.VARCHAR,
    +          SqlTypeName.BOOLEAN, SqlTypeName.DATE, SqlTypeName.DECIMAL, SqlTypeName.TIME,
SqlTypeName.TIMESTAMP,
    +          SqlTypeName.INTERVAL_YEAR_MONTH, SqlTypeName.INTERVAL_DAY_TIME, SqlTypeName.MAP,
SqlTypeName.ARRAY,
    +          SqlTypeName.TINYINT, SqlTypeName.SMALLINT, SqlTypeName.CHAR)
    +      .build();
     
       private boolean contains = false;
     
    +  /**
    +   * Checks if the root portion of the RelNode tree contains a limit 0 pattern.
    +   */
       public static boolean containsLimit0(RelNode rel) {
         FindLimit0Visitor visitor = new FindLimit0Visitor();
         rel.accept(visitor);
         return visitor.isContains();
       }
     
    +  /**
    +   * If all field types of the given node are {@link #TYPES recognized types}, then this
method returns the tree:
    +   *   DrillLimitRel(0)
    +   *     \
    +   *     DrillValuesRel(field types)
    +   * Otherwise, the method returns null.
    +   */
    +  public static DrillRel getValuesRelIfFullySchemaed(final RelNode rel) {
    +    final List<RelDataTypeField> fieldList = rel.getRowType().getFieldList();
    +    final ImmutableList.Builder<RexLiteral> tupleBuilder = new ImmutableList.Builder<>();
    +    final RexBuilder literalBuilder = new RexBuilder(rel.getCluster().getTypeFactory());
    +    for (final RelDataTypeField field : fieldList) {
    +      if (!TYPES.contains(field.getType().getSqlTypeName())) {
    +        return null;
    +      } else {
    +        tupleBuilder.add((RexLiteral) literalBuilder.makeLiteral(null, field.getType(),
false));
    +      }
    +    }
    +
    +    final ImmutableList<ImmutableList<RexLiteral>> tuples = new ImmutableList.Builder<ImmutableList<RexLiteral>>()
    +        .add(tupleBuilder.build())
    +        .build();
    +    final RelTraitSet traits = rel.getTraitSet().plus(DrillRel.DRILL_LOGICAL);
    +    // TODO: ideally, we want the limit to be pushed into values
    +    final DrillValuesRel values = new DrillValuesRel(rel.getCluster(), rel.getRowType(),
tuples, traits);
    --- End diff --
    
    I did this to avoid logical transformation completely, as this is an exceptional case.
    
    LogicalValues [allows an empty list of tuples](https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/logical/LogicalValues.java#L101)
but as Jacques pointed out Drill does not handle that well (we use data to encode types).


> Hive query hangs with limit 0 clause
> ------------------------------------
>
>                 Key: DRILL-3623
>                 URL: https://issues.apache.org/jira/browse/DRILL-3623
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Hive
>    Affects Versions: 1.1.0
>         Environment: MapR cluster
>            Reporter: Andries Engelbrecht
>            Assignee: Sudheesh Katkam
>             Fix For: Future
>
>
> Running a select * from hive.table limit 0 does not return (hangs).
> Select * from hive.table limit 1 works fine
> Hive table is about 6GB with 330 files with parquet using snappy compression.
> Data types are int, bigint, string and double.
> Querying directory with parquet files through the DFS plugin works fine
> select * from dfs.root.`/user/hive/warehouse/database/table` limit 0;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message