flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Muhammad Imran Tariq (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8008) PojoTypeInfo should sort fields fields based on boolean
Date Wed, 08 Nov 2017 04:40:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243362#comment-16243362
] 

Muhammad Imran Tariq commented on FLINK-8008:
---------------------------------------------

I am calling 
_public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields)_ constructor
of class _PojoTypeInfo_.

Below is my code.
_//create a PojoTupeInfo
PojoTypeInfo<Person> sourceType = new PojoTypeInfo<Person>(Person.class, fieldList);
//create a dataset
DataSet<Person> data= env.createInput(new PojoCsvInputFormat<Person>(new Path(textPath),
	CsvInputFormat.DEFAULT_LINE_DELIMITER, CsvInputFormat.DEFAULT_FIELD_DELIMITER, sourceType),
	sourceType);

//create a table of this dataset
Table newT = tableEnv.fromDataSet(text);
//sink table
TableSink sink = new CsvTableSink("D:\\invesco\\POC\\Flink\\rules implementation\\data3.csv",
"|", 1,
		WriteMode.OVERWRITE);
newT.writeToSink(sink);_

So as I said earlier that there are two fields in my POJO class. first is ID(Integer), second
one is Age(Double). PojoTypeInfo info sorts fields in alphabetical order. But CSVReader reads
file and did not sort columns. When I sink my table, then datatype of Age field(which is Double)
get applied on ID field. So initially my data in CSV was:
1,25
2,33
After sink it becomes
1.0,25
2.0,33

To avoid this I want PojoTypeInfo class not to sort fields inside its constructor.






> PojoTypeInfo should sort fields fields based on boolean
> -------------------------------------------------------
>
>                 Key: FLINK-8008
>                 URL: https://issues.apache.org/jira/browse/FLINK-8008
>             Project: Flink
>          Issue Type: Improvement
>          Components: DataSet API
>    Affects Versions: 1.3.2
>            Reporter: Muhammad Imran Tariq
>            Priority: Minor
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> Flink PojoTypeInfo<T> sorts fields array that are passed into constructor arguments.
I want to create another constructor that takes boolean parameter to sort field or not.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message