hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Hammerbacher (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-823) Hadoop Metadata Service
Date Fri, 05 Jun 2009 18:55:07 GMT

    [ https://issues.apache.org/jira/browse/PIG-823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716722#action_12716722
] 

Jeff Hammerbacher commented on PIG-823:
---------------------------------------

Hey Alan,

Thanks for the additional detail. I suppose I can wait for the document to be released to
the public, but it sounds as if you're creating a separate "extended attributes" service to
host non-core file and directory metadata separately from the NN. It's not clear to me that
this is a positive development for Hadoop. Perhaps we should spend the engineering effort
on a single, partitioned, available metadata service for all file and directory attributes?
The project has larger scope and requires but is potentially a cleaner solution for the long
term.

Later,
Jeff

> Hadoop Metadata Service
> -----------------------
>
>                 Key: PIG-823
>                 URL: https://issues.apache.org/jira/browse/PIG-823
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Olga Natkovich
>
> This JIRA is created to track development of a metadata system for  Hadoop. The goal
of the system is to allow users and applications to register data stored on HDFS, search for
the data available on HDFS, and associate metadata such as schema, statistics, etc. with a
particular data unit or a data set stored on HDFS. The initial goal is to provide a fairly
generic, low level abstraction that any user or application on HDFS can use to store an retrieve
metadata. Over time a higher level abstractions closely tied to particular applications or
tools can be developed.
> Over time, it would make sense for the metadata service to become a subproject within
Hadoop. For now, the proposal is to make it a contrib to Pig since Pig SQL is likely to be
the first user of the system.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message