spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From viirya <...@git.apache.org>
Subject [GitHub] spark pull request #19943: [SPARK-16060][SQL] Support Vectorized ORC Reader
Date Tue, 12 Dec 2017 00:17:26 GMT
Github user viirya commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19943#discussion_r156239868
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala
---
    @@ -139,15 +146,25 @@ class OrcFileFormat
           }
         }
     
    +    val resultSchema = StructType(requiredSchema.fields ++ partitionSchema.fields)
    +    val enableVectorizedReader = sparkSession.sessionState.conf.orcVectorizedReaderEnabled
&&
    +      supportBatch(sparkSession, resultSchema)
    --- End diff --
    
    Whether enabling vectorized reader is not the same as supporting batch. You can enable
vectorized reader but not support batch like `ParquetFileFormat` does.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message