hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-509) Redesisn DataNode volumeMap to include all types of Replicas
Date Thu, 06 Aug 2009 01:43:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12739864#action_12739864
] 

Konstantin Shvachko commented on HDFS-509:
------------------------------------------

This is a very complicated hierarchy of classes.
{code}
Block
	ReplicaInfo
		TemporaryReplica
		ReplicaUnderRecovery
		PersistentReplica
			FinalizedReplica
			ReplicaWaitingToBeRecovered
			ReplicaBeingWritten
{code}

I propose to simplify the hierarchy by introducing a ReplicaModificationState{FINALIZED, RBW,
RWR, RUR, TEMPORARY} as a field in ReplicaInfo, which identifies the state of the replica
while it is going through modifications.
So the hierarchy will look like this
{code}
Block
	ReplicaInfo
		ReplicaUnderRecovery
		ReplicaBeingWritten (?)
{code}

# The new ReplicaInfo essentially implements everything you did for PersistentReplica in your
patch. In addition it includes the replicaState field.
# getReadLen() is calculated based on the replicaState, same with isFinalized().
# You do not need to use {{instanceof}} in order to check the type of the replica, just call
getState() instead.
# FinalizedReplica and ReplicaWaitingToBeRecovered are different only in getReadLen() implementation,
so they are done.
# ReplicaUnderRecovery should keep the state of the replica as it was before the recovery.
So you will need some new fields and methods to retrieve old values.
# TemporaryReplica is mainly the same as the ReplicaBeingWritten except for the replicaState.
# The two interfaces PersistentReplicaInfo and BeingWrittenReplicaInfo will probably not be
necessary after that.
# ReplicaBeingWritten has a list of threads in your patch. I do not understand whether you
want to let multiple clients write to the same replica. Why do we need that?
# I'd propose to rename getReadLen() to getVisibleLength() to comply with the design document.



> Redesisn DataNode volumeMap to include all types of Replicas
> ------------------------------------------------------------
>
>                 Key: HDFS-509
>                 URL: https://issues.apache.org/jira/browse/HDFS-509
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: data-node
>    Affects Versions: Append Branch
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: Append Branch
>
>         Attachments: replicas.patch
>
>
> With the new append design, replicas at a DataNode could be in one of the five states:
Finalized, ReplicaBeingWritten, ReplicaWaitingToBeRecovered, ReplicaUnderRecovery, and Temporay.
This jira is to
> 1. define the in-memory representation of these five types of replicas
> 2. redefine volumeMap to include all five types of replicas and remove ongoingCreates
map.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message