spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-23043) Upgrade json4s-jackson to 3.5.3
Date Sat, 13 Jan 2018 15:42:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-23043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Owen reassigned SPARK-23043:
---------------------------------

    Assignee: Takako Shimamoto
    Priority: Minor  (was: Major)

> Upgrade json4s-jackson to 3.5.3
> -------------------------------
>
>                 Key: SPARK-23043
>                 URL: https://issues.apache.org/jira/browse/SPARK-23043
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build
>    Affects Versions: 2.2.1
>            Reporter: Takako Shimamoto
>            Assignee: Takako Shimamoto
>            Priority: Minor
>             Fix For: 2.4.0
>
>
> Spark tried to upgrade json4s several times, but failed and still use a few years old
version 3.2.11. I hope the situation will improve soon.
> h3. Migration from 3.2.11
> Changes that affect Spark are as follows.
> h4. scalap dependency
> Since 3.3, json4s has its own scalap fork and doesn't depend on scala-lang's scalap.
> https://github.com/json4s/json4s/issues/108
> Whereas Spark has added a scala-lang's scalap dependency in POM by the following issue.
> https://github.com/apache/spark/pull/480
> We need to remove scala-lang's scalap dependency for json4s 3.5.3 (same solution as in
json4s).
> h4. The behavior of \ function has changed
> Please see https://github.com/json4s/json4s/pull/309
> h5. a single element
> {code}
> val json =
>       """[ {
>         |  "jobId" : 0,
>         |  "status" : "SUCCEEDED"
>         |} ]""".stripMargin
> val ast = JsonMethods.parse(json)
> {code}
> * version 3.2.11
> {code}
> ast \ "status"  // => JString(SUCCEEDED)
> ast \\ "status" // => JString(SUCCEEDED)
> {code}
> * version 3.5.3
> {code}
> ast \ "status"  // => JArray(List(JString(SUCCEEDED)))
> ast \\ "status" // => JString(SUCCEEDED)
> {code}
> h5. more than a single elements
> {code}
> val json =
>   """[ {
>     |  "jobId" : 1,
>     |  "status" : "FAILED"
>     |}, {
>     |  "jobId" : 0,
>     |  "status" : "SUCCEEDED"
>     |} ]""".stripMargin
> val ast = JsonMethods.parse(json)
> {code}
> * version 3.2.11
> {code}
> ast \ "status"  // => JArray(List(JString(FAILED), JString(SUCCEEDED)))
> ast \\ "status" // => JObject(List((status,JString(FAILED)), (status,JString(SUCCEEDED))))
> {code}
> * version 3.5.3
> {code}
> ast \ "status"  // => JArray(List(JString(FAILED), JString(SUCCEEDED)))
> ast \\ "status" // => JObject(List((status,JString(FAILED)), (status,JString(SUCCEEDED))))
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message