spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ueshin <...@git.apache.org>
Subject [GitHub] spark pull request #19769: [SPARK-12297][SQL] Adjust timezone for int96 data...
Date Fri, 08 Dec 2017 05:10:39 GMT
Github user ueshin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19769#discussion_r155703789
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala
---
    @@ -363,9 +370,25 @@ class ParquetFileFormat
               fileSplit.getLocations,
               null)
     
    +      val sharedConf = broadcastedHadoopConf.value.value
    +      // PARQUET_INT96_TIMESTAMP_CONVERSION says to apply timezone conversions to int96
timestamps'
    +      // *only* if the file was created by something other than "parquet-mr", so check
the actual
    +      // writer here for this file.  We have to do this per-file, as each file in the
table may
    +      // have different writers.
    +      def isCreatedByParquetMr(): Boolean = {
    --- End diff --
    
    nit: Do we need to define this as `def` instead of `val`?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message