hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-222) Support for concatenating of files into a single file
Date Mon, 05 Oct 2009 19:28:31 GMT

    [ https://issues.apache.org/jira/browse/HDFS-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762324#action_12762324
] 

Hairong Kuang commented on HDFS-222:
------------------------------------

Some initial comments:
* ClientProtocol.java:
*# the protocol's version should be bumped;
*# unnecessary changes to the "rename" signature.
* FSNamesytem.java:
*# I would suggest the following changes to the code organization so the method naming is
consistent with existing namespace changes
       concat: an un-synchronous method which contains non-inode related checks on the input
parameters, calls concatInternal, and sync edit log;
       concatInternal: an synchrounous private method which does the real work;
       remove unprotectedConcat in FsNamesystem and add a method "concat" to FSDirectory which
performs all inode-related checkings and namespace changes.
*# permission checking: I would prefer to perform permission checking on target and srcs in
one place. We need WRITE permission on the parent of the source node not on the ancestor.
*# Block size checking could be simplified by making all files have the same preferred block
size and each file's last block is full except for the last file.
*# INodeFile means this inode represents a file. So checking if an inode is a directory should
be performed before converting an inode to be INodeFile.
* FSEditLog.java: since the edit log has a new op, on-disk layout version should be updated.
* minor: should make all concat related methods have the same signatures. Some of them have
"src" as the 2nd parameter. For the first parameter, I prefer to use "target" instead of "trg".


> Support for concatenating of files into a single file
> -----------------------------------------------------
>
>                 Key: HDFS-222
>                 URL: https://issues.apache.org/jira/browse/HDFS-222
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Venkatesh S
>            Assignee: Boris Shkolnik
>         Attachments: HDFS-222-1.patch, HDFS-222-2.patch, HDFS-222-3.patch, HDFS-222-4.patch,
HDFS-222.patch
>
>
> An API to concatenate files of same size and replication factor on HDFS into a single
larger file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message