flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From fhueske <...@git.apache.org>
Subject [GitHub] incubator-flink pull request: enable CSV Reader to ignore invalid ...
Date Thu, 20 Nov 2014 20:38:14 GMT
Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/incubator-flink/pull/201#discussion_r20675590
  
    --- Diff: flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
    @@ -130,6 +132,13 @@ public OUT readRecord(OUT reuse, byte[] bytes, int offset, int numBytes)
{
     			numBytes--;
     		}
     		
    +		//help to prevent null pointer exception in the case of invalid input
    --- End diff --
    
    That's what I meant by letting the DelimitedInputFormat handling invalid lines. I would
not give the null value back to the DataSourceTask, but instead let hte DelimitedInputFormat
catch this and try to call `readRecord()` until a valid record is return and hand that to
the DataSourceTask.
    I am actually surprised that giving a null value to the data source does not cause a NPE.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message