spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From viirya <...@git.apache.org>
Subject [GitHub] spark pull request #20277: [SPARK-23090][SQL] polish ColumnVector
Date Fri, 19 Jan 2018 04:43:27 GMT
Github user viirya commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20277#discussion_r162536698
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala
---
    @@ -50,7 +50,14 @@ private[sql] trait ColumnarBatchScan extends CodegenSupport {
           dataType: DataType,
           nullable: Boolean): ExprCode = {
         val javaType = ctx.javaType(dataType)
    -    val value = ctx.getValue(columnVar, dataType, ordinal)
    +    val value = if (dataType.isInstanceOf[StructType]) {
    +      // `ColumnVector.getStruct` is different from `InternalRow.getStruct`, it only
takes an
    +      // `ordinal` parameter.
    +      s"$columnVar.getStruct($ordinal)"
    +    } else {
    +      ctx.getValue(columnVar, dataType, ordinal)
    +    }
    --- End diff --
    
    Can't we use this API?
    ```scala
      /**
       * Returns the specialized code to access a value from a column vector for a given `DataType`.
       */
      def getValue(vector: String, rowId: String, dataType: DataType): String = {
        ...
      }
    ```


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message