hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-898) Sequential generation of block ids
Date Wed, 20 Jan 2010 22:23:54 GMT

    [ https://issues.apache.org/jira/browse/HDFS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803024#action_12803024
] 

Konstantin Shvachko commented on HDFS-898:
------------------------------------------

You are absolutely right if everything goes well generation stamps help recognizing prehistoric
replicas of newly created blocks.
When I say the problem is relevant I mean to emphasize two cases:
# In case of corrupt blocks, when all generation stamps of its replicas are less than the
block's gen. stamp on the NN, we do not have means to recognize whether the replica is just
a stale replica of this particular block or some other (prehistoric) block. This happens when
you don't have up to date  replicas of the block and a data-node with a prehistoric replica
of previously removed block with the same id starts up.
# The really old blocks, created before introduction of generation stamps, can still cause
the problem.

I agree this is not exactly the problem as you defined it, but they are still related problems.

> Sequential generation of block ids
> ----------------------------------
>
>                 Key: HDFS-898
>                 URL: https://issues.apache.org/jira/browse/HDFS-898
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.20.1
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.22.0
>
>
> This is a proposal to replace random generation of block ids with a sequential generator
in order to avoid block id reuse in the future.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message