flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1512) Add CsvReader for reading into POJOs.
Date Wed, 11 Mar 2015 11:44:38 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356765#comment-14356765
] 

ASF GitHub Bot commented on FLINK-1512:
---------------------------------------

Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/flink/pull/426#discussion_r26205234
  
    --- Diff: flink-java/src/main/java/org/apache/flink/api/java/io/CsvInputFormat.java ---
    @@ -152,6 +177,38 @@ public void setFields(boolean[] sourceFieldMask, Class<?>[]
fieldTypes) {
     	public Class<?>[] getFieldTypes() {
     		return super.getGenericFieldTypes();
     	}
    +
    +	public void setFieldsMap(String[] fieldsMap) {
    +		Preconditions.checkNotNull(fieldsMap);
    +		Preconditions.checkState(typeInformation instanceof PojoTypeInfo);
    +
    +		PojoTypeInfo<OUT> pojoTypeInfo = (PojoTypeInfo<OUT>) typeInformation;
    +
    +		String[] fields = pojoTypeInfo.getFieldNames();
    +		Class<?>[] fieldTypes = getFieldTypes();
    +		this.fieldsMap = Arrays.copyOfRange(fieldsMap, 0, fieldsMap.length);
    +
    +		boolean[] includeMask = new boolean[fieldsMap.length];
    --- End diff --
    
    The ``includeMask`` refers to the fields in the CsvFile and allows to skip fields of the
file.
    For example if a line in your file looks like:
    ``Sam,Smith,09-15-1963,123.123``and you only want to read the first name and the date
field, you would set the ``includeMask`` to ``[true, false, true]`` (missing fields are treated
as ``false``). 
    So the ``includeMask`` should not depend on the ``fieldsMap``, but the number of ``true``
entries in the ``includeMask`` must be equal to the number for fields in the ``fieldsMap``.


> Add CsvReader for reading into POJOs.
> -------------------------------------
>
>                 Key: FLINK-1512
>                 URL: https://issues.apache.org/jira/browse/FLINK-1512
>             Project: Flink
>          Issue Type: New Feature
>          Components: Java API, Scala API
>            Reporter: Robert Metzger
>            Assignee: Chiwan Park
>            Priority: Minor
>              Labels: starter
>
> Currently, the {{CsvReader}} supports only TupleXX types. 
> It would be nice if users were also able to read into POJOs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message