flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From fhueske <...@git.apache.org>
Subject [GitHub] flink issue #2810: [FLINK-3848] Add ProjectableTableSource interface and tra...
Date Mon, 05 Dec 2016 15:25:29 GMT
Github user fhueske commented on the issue:

    https://github.com/apache/flink/pull/2810
  
    This is a good question @tonycox. First of all, we need to agree on the semantics of the
`ProjectableTableSource.setProjection()` method. IMO, the table source must return a `DataSet`
(or `DataStream`) with the fields in the order requested by `setProjection()`. If the `InputFormat`
(or `SourceFunction`) is not able to control the order of fields, the table source needs to
add a `MapFunction` to fix the order. This would happen in `BatchTableSource.getDataSet()`
(or `StreamTableSource.getDataStream()` and hence be opaque to Calcite and not involve a `LogicalCalc`.
    
    From a performance point of view I agree with @wuchong that it would be good to have correct
records produced by the `RowCsvInputFormat`. However, I would see this as a "nice to have"
feature. If you do not want to include it, it would be nice if you could open a JIRA to address
this issue at some later point in time.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message