drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4653) Malformed JSON should not stop the entire query from progressing
Date Thu, 16 Jun 2016 21:55:05 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334765#comment-15334765
] 

ASF GitHub Bot commented on DRILL-4653:
---------------------------------------

Github user ssriniva123 commented on a diff in the pull request:

    https://github.com/apache/drill/pull/518#discussion_r67431369
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
---
    @@ -189,39 +191,33 @@ private long currentRecordNumberInFile() {
       public int next() {
         writer.allocate();
         writer.reset();
    -
         recordCount = 0;
         ReadState write = null;
    -//    Stopwatch p = new Stopwatch().start();
    -    try{
    -      outside: while(recordCount < DEFAULT_ROWS_PER_BATCH) {
    -        writer.setPosition(recordCount);
    -        write = jsonReader.write(writer);
    -
    -        if(write == ReadState.WRITE_SUCCEED) {
    -//          logger.debug("Wrote record.");
    -          recordCount++;
    -        }else{
    -//          logger.debug("Exiting.");
    -          break outside;
    -        }
    -
    +    outside: while(recordCount < DEFAULT_ROWS_PER_BATCH){
    +    try
    +      {
    +            writer.setPosition(recordCount);
    --- End diff --
    
    Aman,
    maven checkstyle:checkstyle did not report any errors before I did my last
    check in. I have changed to reflect 2 spaces for indendation.
    
    On Thu, Jun 16, 2016 at 2:22 PM, Aman Sinha <notifications@github.com>
    wrote:
    
    > In
    > exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
    > <https://github.com/apache/drill/pull/518#discussion_r67426381>:
    >
    > > -      outside: while(recordCount < DEFAULT_ROWS_PER_BATCH) {
    > > -        writer.setPosition(recordCount);
    > > -        write = jsonReader.write(writer);
    > > -
    > > -        if(write == ReadState.WRITE_SUCCEED) {
    > > -//          logger.debug("Wrote record.");
    > > -          recordCount++;
    > > -        }else{
    > > -//          logger.debug("Exiting.");
    > > -          break outside;
    > > -        }
    > > -
    > > +    outside: while(recordCount < DEFAULT_ROWS_PER_BATCH){
    > > +    try
    > > +      {
    > > +            writer.setPosition(recordCount);
    >
    > seems this is still doing indent of 4. We use 2 spaces (see
    > https://drill.apache.org/docs/apache-drill-contribution-guidelines/
    > scroll down to Step 2). Did it pass the mvn command line build without
    > checkstyle violations ?
    >
    > —
    > You are receiving this because you commented.
    > Reply to this email directly, view it on GitHub
    > <https://github.com/apache/drill/pull/518/files/56a16fe3f2bbd1554d65cef6faeaeb63ba41f9a2#r67426381>,
    > or mute the thread
    > <https://github.com/notifications/unsubscribe/ABsaHw_13TKQS-3mIEDhxkCiglpCBA3Sks5qMb6IgaJpZM4IzZkt>
    > .
    >



> Malformed JSON should not stop the entire query from progressing
> ----------------------------------------------------------------
>
>                 Key: DRILL-4653
>                 URL: https://issues.apache.org/jira/browse/DRILL-4653
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - JSON
>    Affects Versions: 1.6.0
>            Reporter: subbu srinivasan
>             Fix For: 1.7.0
>
>
> Currently Drill query terminates upon first encounter of a invalid JSON line.
> Drill has to continue progressing after ignoring the bad records. Something 
> similar to a setting of (ignore.malformed.json) would help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message