incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <>
Subject HCatalog feature wish list
Date Fri, 29 Apr 2011 16:39:24 GMT

We're starting to look into HCatalog to see if it can help us organize
and catalog our data/schemas owned by various groups across our
organization. As a result I have a few questions about some
functionality that I don't think exists yet, but I could be mistaken
(it's been a while since I last worked with Hive).

If these aren't currently supported, would there be interest in
including these features in the roadmap? If so we might be able to
contribute resources to help implement some of them.

- Custom table and field metadata
Is it possible to annotate a table or columns with custom key/value
metadata (i.e. table POCs, descriptions, column data formats, etc)?
This Howl wiki ( had a one-liner about
custom table metadata, but that's the only reference I've been able to
find about this.

- Support for non-RDMS metadata storage
We have groups that currently store a custom schema definition file in
HDFS along side of their actual data. Would it be possible to direct
HCatalog to consult this file instead of the DB for the schema info
for this class of tables?

- Web UI
Are there any know web UIs (or plans for one) to expose and even
modify HCatalog data? We'd like to build a web UI that would help with
data discovery. IIRC, facebook had something similar at one point.

- Support for columnar DBs (i.e., HBase)
I know this doesn't exist currently, but is this something being
considered or requested? HCatalog is focused on fixed-width schemas,
so this would be tricky to represent, but it seems worth exploring.
There seems to be an emerging need to effectively manage and
understand the schemas of such schema-less data stores. :)


View raw message