spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maxim Gekk (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-26376) Skip inputs without tokens by JSON datasource
Date Sat, 15 Dec 2018 16:05:00 GMT
Maxim Gekk created SPARK-26376:
----------------------------------

             Summary: Skip inputs without tokens by JSON datasource
                 Key: SPARK-26376
                 URL: https://issues.apache.org/jira/browse/SPARK-26376
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.4.0
            Reporter: Maxim Gekk


The changes https://github.com/apache/spark/commit/38628dd1b8298d2686e5d00de17c461c70db99a8
can potentially break existing application if it doesn't expect a bad record for string without
any JSON tokens in the PERMISSIVE mode. This ticket aims to return previous behaviour of JSON
datasource and ignore such strings (including empty strings). The from_json function should
keep new behaviour and produce bad records for empty strings and strings without any JSON
tokens.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message