hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-51) per-file replication counts
Date Fri, 07 Apr 2006 23:29:25 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-51?page=all ]

Konstantin Shvachko updated HADOOP-51:
--------------------------------------

    Attachment: Replication.patch

Here is rather big patch. The changes are.

- Create methods include new parameter "short replication"
- If replication is not specified the default replication is used.
- The namenode stores and maintains replication for each file separately.
- File replication can be obtained from the namenode as a part of DFSFileInfo.
- 2 new namenode config parameters
   dfs.replication.max
   dfs.replication.min
which are checked when a new file is created.
- Namenode image and edit log file format are modified. Both contain version
number at the beginning now. The versions are negative. I started from version -1.
When the namenode starts you should expect that your current dfs image will be
loaded and converted into new format. All old files will have the same default
replication equal to the value of dfs.replication of your config.



> per-file replication counts
> ---------------------------
>
>          Key: HADOOP-51
>          URL: http://issues.apache.org/jira/browse/HADOOP-51
>      Project: Hadoop
>         Type: New Feature

>   Components: dfs
>     Versions: 0.2
>     Reporter: Doug Cutting
>     Assignee: Konstantin Shvachko
>      Fix For: 0.2
>  Attachments: Replication.patch
>
> It should be possible to specify different replication counts for different files.  Perhaps
an option when creating a new file should be the desired replication count.  MapReduce should
take advantage of this feature so that job.xml and job.jar files, which are frequently accessed
by lots of machines, are more highly replicated than large data files.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message