hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1245) Plugable block id generation
Date Sun, 13 Jan 2013 13:22:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13552195#comment-13552195

Hudson commented on HDFS-1245:

Integrated in Hadoop-Mapreduce-trunk #1312 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1312/])
    HDFS-1245. Change typo in Pluggable. (Revision 1432542)
HDFS-1245. Plugable block id generation. Contributed by Konstantin Shvachko. (Revision 1432539)

     Result = FAILURE
shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1432542
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1432539
Files : 
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/IdGenerator.java
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SequentialNumber.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/RandomBlockIdGenerator.java

> Plugable block id generation 
> -----------------------------
>                 Key: HDFS-1245
>                 URL: https://issues.apache.org/jira/browse/HDFS-1245
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: namenode
>    Affects Versions: 0.22.0
>            Reporter: Dmytro Molkov
>            Assignee: Konstantin Shvachko
>             Fix For: 2.0.3-alpha
>         Attachments: blockIdGenerator-branch2.patch, blockIdGenerator.patch
> The idea is to have a way to easily create block id generation engines that may fit a
certain purpose. One of them could be HDFS-898 started by Konstantin, but potentially others.
> We chatted with Dhruba about this for a while and came up with the following approach:
> There should be a BlockIDGenerator interface that has following methods:
> void blockAdded(Block)
> void blockRemoved(Block)
> Block nextBlock()
> First two methods are needed for block generation engines that hold a certain state.
During the restart, when namenode reads the fsimage it will notify generator about all the
blocks it reads from the image and during runtime namenode will notify the generator about
block removals on file deletion.
> The instance of the generator will also have a reference to the block registry, the interface
that BlockManager implements. The only method there is __blockExists(Block)__, so that the
current random block id generation can be implemented, since it needs to check with the block
manager if the id is already present.
> What does the community think about this proposal?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message