drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4653) Malformed JSON should not stop the entire query from progressing
Date Thu, 14 Jul 2016 19:01:20 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378138#comment-15378138
] 

ASF GitHub Bot commented on DRILL-4653:
---------------------------------------

Github user chunhui-shi commented on a diff in the pull request:

    https://github.com/apache/drill/pull/518#discussion_r70863798
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
---
    @@ -110,21 +118,29 @@ public void ensureAtLeastOneField(ComplexWriter writer) {
             emptyStatus.set(i, true);
           }
           if (i == 0 && !allTextMode) {
    -        // when allTextMode is false, there is not much benefit to producing all the
empty
    -        // fields; just produce 1 field.  The reason is that the type of the fields is
    -        // unknown, so if we produce multiple Integer fields by default, a subsequent
batch
    -        // that contains non-integer fields will error out in any case.  Whereas, with
    -        // allTextMode true, we are sure that all fields are going to be treated as varchar,
    -        // so it makes sense to produce all the fields, and in fact is necessary in order
to
    +        // when allTextMode is false, there is not much benefit to producing all
    --- End diff --
    
    Seems the line width changed here(line 121-132). Please reorganize the text. If possible
keep the original text unchanged. The same in line 140-143


> Malformed JSON should not stop the entire query from progressing
> ----------------------------------------------------------------
>
>                 Key: DRILL-4653
>                 URL: https://issues.apache.org/jira/browse/DRILL-4653
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - JSON
>    Affects Versions: 1.6.0
>            Reporter: subbu srinivasan
>             Fix For: Future
>
>
> Currently Drill query terminates upon first encounter of a invalid JSON line.
> Drill has to continue progressing after ignoring the bad records. Something 
> similar to a setting of (ignore.malformed.json) would help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message