flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2168) Add HBaseTableSource
Date Wed, 01 Feb 2017 15:41:51 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848530#comment-15848530

ASF GitHub Bot commented on FLINK-2168:

Github user wuchong commented on the issue:

    Sorry for the late response. 
    Regarding to the `HBaseTableSchema`, I agree with that to move the `addColumn(...)` method
into `HBaseTableSource`. 
    Regarding to the nested vs flat schema, I prefer the nested schema. It is more intuitive
to use.
    As for the nested schema doesn't support to push projections down, I think we should extend
`ProjectableTableSource` to support push projections down to a composite type. We can keep
the interface unchanged, i.e. `def projectFields(fields: Array[Int]): ProjectableTableSource[T]`.
But the index of `fields` should be the flat index. We can use the flat field indexes to do
projection pushdown even if it is a nested schema.
    For example, a table source with schema `a: Int, b: Row<b1: String, b2: Int>, c:
Boolean`, the flat indexes of `a, b.b1, b.b2, c` are `0, 1, 2, 3`. So a project `SELECT b.b1,
c FROM T` will result a `fields` `Array(1,3)`.
    What do you think ? 
    For me the biggest drawback of a nested schema is the lacking support to push projections

> Add HBaseTableSource
> --------------------
>                 Key: FLINK-2168
>                 URL: https://issues.apache.org/jira/browse/FLINK-2168
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>    Affects Versions: 0.9
>            Reporter: Fabian Hueske
>            Assignee: ramkrishna.s.vasudevan
>            Priority: Minor
> Add a {{HBaseTableSource}} to read data from a HBase table. The {{HBaseTableSource}}
should implement the {{ProjectableTableSource}} (FLINK-3848) and {{FilterableTableSource}}
(FLINK-3849) interfaces.
> The implementation can be based on Flink's {{TableInputFormat}}.

This message was sent by Atlassian JIRA

View raw message