flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maximilian Michels (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1872) How can generation dataset in flink automatic depend on number of filed and data type
Date Tue, 14 Apr 2015 11:36:13 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493949#comment-14493949
] 

Maximilian Michels commented on FLINK-1872:
-------------------------------------------

If you want to automatically generate the type annotations, you'll have to write a parser
which extracts the correct types for the fields in your CSV files. It's not possible to write
a generalized parser for all CSV files because types can't always be inferred unambiguously.
That's why Flink leaves it up for the user to decide how the CSV file should be parsed.

> How can generation dataset in flink automatic depend on number of filed and data type

> --------------------------------------------------------------------------------------
>
>                 Key: FLINK-1872
>                 URL: https://issues.apache.org/jira/browse/FLINK-1872
>             Project: Flink
>          Issue Type: Bug
>            Reporter: hagersaleh
>
> when read csv file want generate dataset function automatic
> Example write this Mnola
> but want generate this code automatic for any csv flie input
> final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
>  
> DataSet<Tuple5<Integer, String, String, Integer, Double>> customers = getCustomerDataSet(env);
> private static DataSet<Tuple5<Integer, String, String, Integer, Double>>
getCustomerDataSet(ExecutionEnvironment env) {
> 		return env.readCsvFile(customerPath)
> 					.fieldDelimiter("|")
> 					.includeFields("11110100")
> 					.types(Integer.class, String.class, String.class, Integer.class, Double.class);
> 	}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message