spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rxin <...@git.apache.org>
Subject [GitHub] spark pull request #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json...
Date Fri, 26 Oct 2018 00:00:01 GMT
Github user rxin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/22775#discussion_r228372331
  
    --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
---
    @@ -770,8 +776,17 @@ case class SchemaOfJson(
         factory
       }
     
    -  override def convert(v: UTF8String): UTF8String = {
    -    val dt = Utils.tryWithResource(CreateJacksonParser.utf8String(jsonFactory, v)) {
parser =>
    +  @transient
    +  private lazy val json = child.eval().asInstanceOf[UTF8String]
    --- End diff --
    
    It's not weird that users want to use schema_of_json at all.
    
    Imagine it's a very large json with very complicated string. It's pretty difficult to
actually write the ddl string.



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message