flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1466) Add InputFormat to read from (Hive) tables stored in MetaStore
Date Sat, 31 Jan 2015 11:50:34 GMT
Fabian Hueske created FLINK-1466:

             Summary: Add InputFormat to read from (Hive) tables stored in MetaStore
                 Key: FLINK-1466
                 URL: https://issues.apache.org/jira/browse/FLINK-1466
             Project: Flink
          Issue Type: New Feature
          Components: Java API, Scala API
            Reporter: Fabian Hueske
            Assignee: Fabian Hueske
            Priority: Minor

Hive features a table catalog called MetaStore to store information about tables. This information

- Table schema (field names, types)
- Storage location in HDFS
- HadoopInputFormat to read the data
- SerDe information to extract fields
- Information about partitions

This JIRA proposes to add an InputFormat to read Hive tables. For a given database and table
name, the IF should access the megastore to get all relevant information to read the data.
Further parameters are field names (selection) and partitions (limited predicates).

For a first version, the IF can be limited to flat tables (Hive supports nested data) with
primitive field types.

This message was sent by Atlassian JIRA

View raw message