hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-4974) Analyze and add annotations to Namenode and Datanode protocol methods to enable retry
Date Wed, 10 Jul 2013 16:13:49 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13704367#comment-13704367
] 

Suresh Srinivas edited comment on HDFS-4974 at 7/10/13 4:11 PM:
----------------------------------------------------------------

Here is the analysis of current Namenode protocols:

h3. Idempotent operations missing @Idempotent annotation:
*HAServiceProtocol*
* monitorHealth()
* transitionToActive()
* transitionToStandby()
* getServiceStatus()

*GetUserMappingsProtocol*
* getGroupsForUser()

*RefreshUserMappingsProtocol*
* refreshUserToGroupsMappings()
* refreshSuperUserGroupsConfiguration()

*RefreshAuthorizationPolicyProtocol*
* refreshServiceAcl()

*NamenodeProtocol*
* getBlocks()
* getBlockKeys()
* getTransactionID()
* getMostRecentCheckpointTxId()
* versionRequest()
* errorReport()
* registerSubordinateNamenode()
* getEditLogManifest()

*DatanodeProtocol*
* registerDatanode()
* sendHeartbeat()
* blockReport()
* blockReceivedAndDeleted() - need to check the code
* errorReport()
* versionRequest()
* reportBadBlocks()

*ClientProtocol*
* restoreFailedStorage()
* refreshNodes()
* finalizeUpgrade()
* metaSave()
* allowSnapshot()
* disallowSnapshot()
* getSnapshotDiffReport()
* abandonBlock() - with sequential block ID change this can be made idempotent
* cancelDelegationToken() - With some change to make it Idempotent
* getDataEncryptionKey() - this should be marked as idempotent

h3. Already correctly marked as Idempotent:
*ClientProtocol*
* getBlockLocations()
* getServiceDefaults()
* setReplication()
* setPermission()
* setOwner()
* addBlock()
* reportBadBlocks()
* getListing()
* getSnapshottableDirListing()
* getStats()
* getDatanodeReport()
* getPreferredBlockSize()
* setSafeMode()
* listCorruptFileBlocks()
* setBalancerBandwidth()
* getFileInfo()
* isFileClosed()
* getFileLinkInfo()
* getContentSummary()
* setQuota()
* fsync()
* setTimes()
* getLinkTarget()
* renewLease() 
* recoverLease()
* rollEdits()
* updateBlockForPipeline()
* getDelegationToken()
* renewDelegationToken()
* getAdditionalDatanode()
* complete()

h3. Non-idempotent requests:
*ClientProtocol*
* create()
* append()
* rename()
* concat()
* rename2()
* delete()
* saveNamespace() - this can be made idempotent, if writeLock is held in this operation
* createSymlink()
* updatePipeline()
* createSnapshot()
* deleteSnapshot()
* renameSnapshot()

*NamenodeProtocol*
* rollEditLog()
* startCheckpoint()
* endCheckpoint()
* commitBlockSynchronized()

h3. Incorrectly marked as Idempotent:
ClientProtocol#mkdirs() - fails on retry attempt if first attempt is successful

                
      was (Author: sureshms):
    Here is the analysis of current Namenode protocols:

h3. Idempotent operations missing @Idempotent annotation:
*HAServiceProtocol*
* monitorHealth()
* transitionToActive()
* transitionToStandby()
* getServiceStatus()

*GetUserMappingsProtocol*
* getGroupsForUser()

*RefreshUserMappingsProtocol*
* refreshUserToGroupsMappings()
* refreshSuperUserGroupsConfiguration()

*RefreshAuthorizationPolicyProtocol*
* refreshServiceAcl()

*NamenodeProtocol*
* getBlocks()
* getBlockKeys()
* getTransactionID()
* getMostRecentCheckpointTxId()
* versionRequest()
* errorReport()
* registerSubordinateNamenode()
* getEditLogManifest()

*DatanodeProtocol*
* registerDatanode()
* sendHeartbeat()
* blockReport()
* blockReceivedAndDeleted() - need to check the code
* errorReport()
* versionRequest()
* reportBadBlocks()

*ClientProtocol*
* restoreFailedStorage()
* refreshNodes()
* finalizeUpgrade()
* metaSave()
* allowSnapshot()
* disallowSnapshot()
* getSnapshotDiffReport()
* abandonBlock() - with sequential block ID change this can be made idempotent
* cancelDelegationToken() - With some change to make it Idempotent
* getDataEncryptionKey() - this should be marked as idempotent

h3. Already correctly marked as Idempotent:
*ClientProtocol*
* getBlockLocations()
* getServiceDefaults()
* setReplication()
* setPermission()
* setOwner()
* addBlock()
* reportBadBlocks()
* getListing()
* getSnapshottableDirListing()
* getStats()
* getDatanodeReport()
* getPreferredBlockSize()
* setSafeMode()
* listCorruptFileBlocks()
* setBalancerBandwidth()
* getFileInfo()
* isFileClosed()
* getFileLinkInfo()
* getContentSummary()
* setQuota()
* fsync()
* setTimes()
* getLinkTarget()
* renewLease() 
* recoverLease()
* rollEdits()
* updateBlockForPipeline()
* getDelegationToken()
* renewDelegationToken()
* getAdditionalDatanode()
* complete()

h3. Non-idempotent requests:
*ClientProtocol*
* create()
* append()
* rename()
* concat()
* rename2()
* delete()
* saveNamespace() - this can be made idempotent, if writeLock is held in this operation
* createSymlink()
* updatePipeline()
* createSnapshot()
* deleteSnapshot()
* renameSnapshot()
*NamenodeProtocol*
* rollEditLog()
* startCheckpoint()
* endCheckpoint()
* commitBlockSynchronized()

h3. Incorrectly marked as Idempotent:
ClientProtocol#mkdirs() - fails on retry attempt if first attempt is successful

                  
> Analyze and add annotations to Namenode and Datanode protocol methods to enable retry
> -------------------------------------------------------------------------------------
>
>                 Key: HDFS-4974
>                 URL: https://issues.apache.org/jira/browse/HDFS-4974
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, namenode
>            Reporter: Suresh Srinivas
>
> This jira is intended for:
> # Discussing current @Idempotent annotations in HDFS protocols and adding that annotation
where it is missing.
> # Discuss how retry should be enabled for non-idempotent requests.
> I will post the analysis of current methods in subsequent comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message