spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gatorsmile <>
Subject [GitHub] spark pull request #20610: [SPARK-23426][SQL] Use `hive` ORC impl and disabl...
Date Wed, 14 Feb 2018 23:50:52 GMT
Github user gatorsmile commented on a diff in the pull request:
    --- Diff: docs/ ---
    @@ -1004,6 +1004,24 @@ Configuration of Parquet can be done using the `setConf` method
on `SparkSession
    +## ORC Files
    +Since Spark 2.3, Spark supports a vectorized ORC reader with a new ORC file format for
ORC files. To do that, the following configurations are newly added. The vectorized reader
is used for the native ORC tables (e.g., the ones created using the clause `USING ORC`) when
`spark.sql.orc.impl` is set to `native` and `spark.sql.orc.enableVectorizedReader` is set
to `true`. For the Hive ORC serde table (e.g., the ones created using the clause `USING HIVE
OPTIONS (fileFormat 'ORC')`), the vectorized reader is used when `spark.sql.hive.convertMetastoreOrc`
is set to `true`.
    --- End diff --
    `table ` -> `tables `


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message