drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4653) Malformed JSON should not stop the entire query from progressing
Date Fri, 10 Jun 2016 23:07:20 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325472#comment-15325472
] 

ASF GitHub Bot commented on DRILL-4653:
---------------------------------------

GitHub user ssriniva123 opened a pull request:

    https://github.com/apache/drill/pull/518

    DRILL-4653.json - Malformed JSON should not stop the entire query from progressing

    https://issues.apache.org/jira/browse/DRILL-4653
    
    - The default is to stop processing as is today when JSON parser encounters an exception
    - Setting store.json.reader.skip_malformed_records will ensure that query progresses after
    skipping the bad records
    - Added two unit tests
    - Also did testing after deploying the new build: Both positive and negative tests were
done.
    - Negative test result:
    org.apache.drill.common.exceptions.UserRemoteException: DATA_READ ERROR: Error parsing
JSON - Unexpected character ('{' (code 123)): was expecting comma to separate OBJECT entries

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ssriniva123/drill master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/drill/pull/518.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #518
    
----
commit 4fc70faf0e5ad5d434b944d084bc0b0e90d41c39
Author: Subbu Srinivasan <ssrinivasan@zscaler.com>
Date:   2016-06-10T22:58:49Z

    Fixes for DRILL-4653.json

----


> Malformed JSON should not stop the entire query from progressing
> ----------------------------------------------------------------
>
>                 Key: DRILL-4653
>                 URL: https://issues.apache.org/jira/browse/DRILL-4653
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - JSON
>    Affects Versions: 1.6.0
>            Reporter: subbu srinivasan
>
> Currently Drill query terminates upon first encounter of a invalid JSON line.
> Drill has to continue progressing after ignoring the bad records. Something 
> similar to a setting of (ignore.malformed.json) would help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message