flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2168) Add HBaseTableSource
Date Wed, 01 Feb 2017 15:41:51 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848530#comment-15848530
] 

ASF GitHub Bot commented on FLINK-2168:
---------------------------------------

Github user wuchong commented on the issue:

    https://github.com/apache/flink/pull/3149
  
    Sorry for the late response. 
    
    Regarding to the `HBaseTableSchema`, I agree with that to move the `addColumn(...)` method
into `HBaseTableSource`. 
    
    Regarding to the nested vs flat schema, I prefer the nested schema. It is more intuitive
to use.
    As for the nested schema doesn't support to push projections down, I think we should extend
`ProjectableTableSource` to support push projections down to a composite type. We can keep
the interface unchanged, i.e. `def projectFields(fields: Array[Int]): ProjectableTableSource[T]`.
But the index of `fields` should be the flat index. We can use the flat field indexes to do
projection pushdown even if it is a nested schema.
    
    For example, a table source with schema `a: Int, b: Row<b1: String, b2: Int>, c:
Boolean`, the flat indexes of `a, b.b1, b.b2, c` are `0, 1, 2, 3`. So a project `SELECT b.b1,
c FROM T` will result a `fields` `Array(1,3)`.
    
    What do you think ? 
    
    
    For me the biggest drawback of a nested schema is the lacking support to push projections
down. 


> Add HBaseTableSource
> --------------------
>
>                 Key: FLINK-2168
>                 URL: https://issues.apache.org/jira/browse/FLINK-2168
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>    Affects Versions: 0.9
>            Reporter: Fabian Hueske
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Minor
>
> Add a {{HBaseTableSource}} to read data from a HBase table. The {{HBaseTableSource}}
should implement the {{ProjectableTableSource}} (FLINK-3848) and {{FilterableTableSource}}
(FLINK-3849) interfaces.
> The implementation can be based on Flink's {{TableInputFormat}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message