spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From viirya <...@git.apache.org>
Subject [GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...
Date Wed, 06 Sep 2017 14:16:28 GMT
Github user viirya commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18875#discussion_r137280095
  
    --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala
---
    @@ -22,24 +22,49 @@ import java.io.Writer
     import com.fasterxml.jackson.core._
     
     import org.apache.spark.sql.catalyst.InternalRow
    +import org.apache.spark.sql.catalyst.analysis.TypeCheckResult
     import org.apache.spark.sql.catalyst.expressions.SpecializedGetters
     import org.apache.spark.sql.catalyst.util.{ArrayData, DateTimeUtils, MapData}
     import org.apache.spark.sql.types._
     
     private[sql] class JacksonGenerator(
    -    schema: StructType,
    +    dataType: DataType,
         writer: Writer,
         options: JSONOptions) {
       // A `ValueWriter` is responsible for writing a field of an `InternalRow` to appropriate
       // JSON data. Here we are using `SpecializedGetters` rather than `InternalRow` so that
       // we can directly access data in `ArrayData` without the help of `SpecificMutableRow`.
       private type ValueWriter = (SpecializedGetters, Int) => Unit
     
    +  // `JackGenerator` only supports to write out a struct, an array of struct or an arbitrary
map
    --- End diff --
    
    `JackGenerator` can only be initialized with a StructType or a MapType. 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message