hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-517) Introduce BlockInfoUnderConstruction to reflect block replica states while writing.
Date Thu, 20 Aug 2009 01:37:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745271#action_12745271
] 

Hairong Kuang commented on HDFS-517:
------------------------------------

Two general comments:
1. GetAdditionalBlock or close should not convert the penultimate block to be Complete. Instead
it should be the block reports that trigger this conversion. GetAdditionalBlock or close could
change the last block to be either Committed or Complete.
2. In an UnderConstruction block, datanode reported finalized replicas are kept in triplets
but the client reported location is kept in locations. It might be better to merge these two
into one list. But I am OK that we wait until we work on block reports.

HdfsConstants.java
1. Comment for Temporary replica: "created for replication only" should be "created for replication
or block move"
2. enum BlockUCState  better to be renamed to BlockState because it includes Complete block.
Change the comment for this type too.

BlockInfo.java
1. why constructors need to be protected?
2. Constructor BlockInfo(BlockInfo) has a risk of NullPointerException if from.inode == null
3. Better rename getBlockUCState() to be getBlockState() or getState().
4. convertToBlockUnderConstruction() should makes sure that this block is complete before
converting. It should initialize pipeline too. Otherwise, this block can not be read before
client updates pipeline.

BlockUnderConstruction.java
1. The 2nd constructor should make sure that the input state is not equal to Complete
2. convertToCompleteBlock(): by design both a Committed block and an UnderConstruction block
can be converted to a Complete block.
3. commitBlock(Block block): If a block has already been committed, can it be commit again?
4. assignPrimaryDatanode() line 138: <= should be <
5. We probably need methods to update each replica's state/len/gs

BlockManager.java
1. commitLastBlock: the method could be renamed to reflect what this method really does. It
commits last block and completes penultimate block.
2. completeBlock(InodeFile, int blkIndex): is it better to change its signature to be completeBlock(InodeFileUnderConstruction,
BlockInfoUnderConstruction)

BlocksMap.java
addINode(BlockInfo, InodeFile) allows a map entry to be replaced, but it does not update the
datanode to blocks map as replaceBlock does.

FSdirectory.java
addToParent: In method signature, is it better to change Block[] to be BlockInfo[]?

FSNamesystem:
nextGenerationStampForBlock(): error message " is already committed, !fileINode.isUnderConstruction()."
should add one more condition: !storedBlock.isUnderConstruction().

INodeFile.java
getLastBlock(): what's the advantage to have the return type to be a <T extends BlockInfo>
T instead of BlockInfo?


> Introduce BlockInfoUnderConstruction to reflect block replica states while writing.
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-517
>                 URL: https://issues.apache.org/jira/browse/HDFS-517
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 0.21.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: Append Branch
>
>         Attachments: Append-UCBlock-h265.patch, Append-UCBlock.patch, Append-UCBlock.patch,
Append-UCBlock.patch
>
>
> Currently when a block is created its locations are stored in {{INodeFileUnderConstruction.targets}},
which correspond to the last allocated block. With the new append design we will need to keep
track of block replicas for several blocks rather than just the last one.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message