spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <>
Subject [jira] [Assigned] (SPARK-20314) Inconsistent error handling in JSON parsing SQL functions
Date Thu, 20 Apr 2017 00:08:04 GMT


Apache Spark reassigned SPARK-20314:

    Assignee: Apache Spark

> Inconsistent error handling in JSON parsing SQL functions
> ---------------------------------------------------------
>                 Key: SPARK-20314
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.1.0
>            Reporter: Eric Wasserman
>            Assignee: Apache Spark
> Most parse errors in the JSON parsing SQL functions (e.g. json_tuple, get_json_object)
will return a null(s) if the JSON is badly formed. However, if Jackson determines that the
string includes invalid characters it will throw an exception (
Invalid UTF-32 character) that Spark does not catch. This creates a robustness problem in
that these functions cannot be used at all when there may be dirty data as these exceptions
will kill the jobs.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message