flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhenqiu Huang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-10729) Create a Hive connector for Hive data access in Flink
Date Fri, 02 Nov 2018 03:23:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672530#comment-16672530

Zhenqiu Huang commented on FLINK-10729:


Agree. If you want to use native formats in Hive, I feel an initial prototype is definitely
needed for me to follow. For the direction of implementation, probably we can align with [~fhueske] also.
Looking forward to the design.

> Create a Hive connector for Hive data access in Flink
> -----------------------------------------------------
>                 Key: FLINK-10729
>                 URL: https://issues.apache.org/jira/browse/FLINK-10729
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API &amp; SQL
>    Affects Versions: 1.6.2
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>            Priority: Major
> As part of Flink-Hive integration effort, it's important for Flink to access (read/write)
Hive data, which is the responsibility of Hive connector. While there is a HCatalog data connector
in the code base, it's not complete (i.e. missing all connector related classes such as validators,
etc.). Further, HCatalog interface has many limitations such as accessing a subset of Hive
data, supporting a subset of Hive data types, etc. In addition, it's not actively maintained.
In fact, it's now only a sub-project in Hive.
> Therefore, here we propose a complete connector set for Hive tables, not via HCatalog,
but via direct Hive interface. HCatalog connector will be deprecated.
> Please note that connector on Hive metadata is already covered in other JIRAs, as {{HiveExternalCatalog}}.

This message was sent by Atlassian JIRA

View raw message