hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Runping Qi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-170) setReplication and related bug fixes
Date Thu, 27 Apr 2006 00:25:03 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-170?page=comments#action_12376614 ] 

Runping Qi commented on HADOOP-170:
-----------------------------------


It will be more effective if after assigning a map task to a specific jobtracker node, the
jobtrackercan request the name node to replicate the blocks of the split for the maptask on
to the jobtracker node.

> setReplication and related bug fixes
> ------------------------------------
>
>          Key: HADOOP-170
>          URL: http://issues.apache.org/jira/browse/HADOOP-170
>      Project: Hadoop
>         Type: Improvement

>   Components: fs, dfs
>     Versions: 0.1.1
>     Reporter: Konstantin Shvachko
>     Assignee: Konstantin Shvachko
>      Fix For: 0.2
>  Attachments: setReplication.patch
>
> Having variable replication (HADOOP-51) it is natural to be able to
> change replication for existing files. This patch introduces the functionality.
> Here is a detailed list of issues addressed by the patch.
> 1) setReplication() and getReplication() methods are implemented.
> 2) DFSShell prints file replication for any listed file.
> 3) Bug fix. FSDirectory.delete() logs delete operation even if it is not successful.
> 4) Bug fix. This is a distributed bug.
> Suppose that file replication is 3, and a client reduces it to 1.
> Two data nodes will be chosen to remove their copies, and will do that.
> After a while they will report to the name node that the copies have been actually deleted.
> Until they report the name node assumes the copies still exist.
> Now the client decides to increase replication back to 3 BEFORE the data nodes
> reported the copies are deleted. Then the name node can choose one of the data nodes,
> which it thinks have a block copy, to replicate the block to new data nodes.
> This setting is quite unusual but possible even without variable replications.
> 5) Logging for name and data nodes is improved in several cases.
> E.g. data nodes never logged that they deleted a block.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message