spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From viirya <>
Subject [GitHub] spark pull request #18875: [SPARK-21513][SQL] Allow UDF to_json support conv...
Date Wed, 06 Sep 2017 14:16:28 GMT
Github user viirya commented on a diff in the pull request:
    --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala
    @@ -22,24 +22,49 @@ import
     import com.fasterxml.jackson.core._
     import org.apache.spark.sql.catalyst.InternalRow
    +import org.apache.spark.sql.catalyst.analysis.TypeCheckResult
     import org.apache.spark.sql.catalyst.expressions.SpecializedGetters
     import org.apache.spark.sql.catalyst.util.{ArrayData, DateTimeUtils, MapData}
     import org.apache.spark.sql.types._
     private[sql] class JacksonGenerator(
    -    schema: StructType,
    +    dataType: DataType,
         writer: Writer,
         options: JSONOptions) {
       // A `ValueWriter` is responsible for writing a field of an `InternalRow` to appropriate
       // JSON data. Here we are using `SpecializedGetters` rather than `InternalRow` so that
       // we can directly access data in `ArrayData` without the help of `SpecificMutableRow`.
       private type ValueWriter = (SpecializedGetters, Int) => Unit
    +  // `JackGenerator` only supports to write out a struct, an array of struct or an arbitrary
    --- End diff --
    `JackGenerator` can only be initialized with a StructType or a MapType. 


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message