hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3307) Archives in Hadoop.
Date Fri, 25 Apr 2008 19:35:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12592481#action_12592481

Doug Cutting commented on HADOOP-3307:

> Why not do an equivalent of NFS mount

On unix, mounts are not managed by a filesystem implementation, but by the kernel.  If we
were to add a mount mechanism, we should add it at the generic FileSystem level, not within

In fact, we already have a mount mechanism, but it only permits mounts by scheme, not at any
point in the path.  We could add a mechanism to mount a filesystem under an arbitrary path,
or even a regex like "*.har".  This could be confusing, however, since a path that looks like
an hdfs: path would really be using some other protocol.  And I don't yet see that we need
to add a new mount feature, since I think the existing one is sufficient to implement this
feature.  Also, if we use "har:" or "hdfs-ar:" then it is clear that these are not normal
HDFS files.

A feature that might be good to add to the generic FileSystem is symbolic links.  Then one
could add a link in HDFS to an archive URI, thus grafting it into the namespace.  If one linked
hdfs://host:port/dir/foo/ to hdfs-ar://host:port/dir/foo.har then one could list the files
in the former to get the uris in the latter.  But that's beyond the scope of this issue. 
This would be good future work to make transparent archives possible.

> Archives in Hadoop.
> -------------------
>                 Key: HADOOP-3307
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3307
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Mahadev konar
>            Assignee: Mahadev konar
>             Fix For: 0.18.0
> This is a new feature for archiving and unarchiving files in HDFS. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message