hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrzej Bialecki (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3307) Archives in Hadoop.
Date Fri, 25 Apr 2008 19:31:56 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12592478#action_12592478

Andrzej Bialecki  commented on HADOOP-3307:


bq. Wouldnt this also require changes to DFS Namenode?

Perhaps not - the way I was thinking about it, this would be a function of a DFS client (i.e.
the FileSystem on the API level). The FileSystem client would be aware of the current mounts
(from Configuration), and for matching path prefixes it would translate file ops to operations
on the archive file retrieved from the underlying configured FileSystem.

This way we don't have to modify the Namenode, we don't have to hack the url schemas, the
URIs are transparent, and all processing load is at the cost of a user task, i.e. we don't
create an additional load on the namenode. Additionally, the scope of the "mount" is the current
configuration, e.g. a job, so it's not a permanent mount, it can be different for each job,
and no cleanup or unmount is needed.

> Archives in Hadoop.
> -------------------
>                 Key: HADOOP-3307
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3307
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Mahadev konar
>            Assignee: Mahadev konar
>             Fix For: 0.18.0
> This is a new feature for archiving and unarchiving files in HDFS. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message