hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2393) Mark appropriate methods of ClientProtocol with the idempotent annotation
Date Thu, 06 Oct 2011 04:36:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121717#comment-13121717
] 

Aaron T. Myers commented on HDFS-2393:
--------------------------------------

I've been working on categorizing all of the 45 operations currently defined in {{ClientProtocol}}.
This is the breakdown of what I've come up with:

Read ops, therefore idempotent:

* getBlockLocations
* getPreferredBlockSize
* getListing
* getFileInfo
* getFileLinkInfo
* getStats
* getDatanodeReport
* listCorruptFileBlocks
* getContentSummary
* getLinkTarget
* getServerDefaults

Write ops that I believe should be marked idempotent:

* setReplication
* setPermission
* setOwner
* reportBadBlocks
* mkdirs (in fact, it looks like this op can never return false. It either returns true or
throws an exception.)
* renewLease
* setQuota
* fsync
* setTimes
* getDelegationToken
* renewDelegationToken

Write ops I'm unsure about:

* getAdditionalDatanode (I'm not sure this one should even be marked OperationCategory.WRITE
- it only gets the readLock)
* recoverLease
* updateBlockForPipeline

Write ops that I believe should *not* be idempotent:

* create
* append
* abandonBlock
* addBlock
* complete
* rename
* concat
* rename2
* delete
* createSymLink
* updatePipeline
* cancelDelegationToken

Ops which are more administrative, and arguably shouldn't be in ClientProtocol:

* setSafeMode
* saveNamespace
* restoreFailedStorage
* refreshNodes
* finalizeUpgrade
* distributedUpgradeProgress
* metaSave
* setBalancerBandwidth

Comments are most welcome. In particular, I'm interested in:

# Does anyone have any insight about the three operations I'm not sure about? We could play
it safe at first, and not mark them idempotent. Doing so will result in a few more client
failures during an NN fail over, but will not result in correctness issues.
# What to do about the 8 operations which seem administrative in nature?
# Did I misidentify any of the write operations which I believe to be idempotent?

Once we hash these things out a little bit, I'll go ahead and make a patch for it.
                
> Mark appropriate methods of ClientProtocol with the idempotent annotation
> -------------------------------------------------------------------------
>
>                 Key: HDFS-2393
>                 URL: https://issues.apache.org/jira/browse/HDFS-2393
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs client
>    Affects Versions: HA branch (HDFS-1623)
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>             Fix For: HA branch (HDFS-1623)
>
>
> HDFS-1973 will make the {{DFSClient}} take advantage of the annotation. This JIRA is
to identify which methods can be marked as idempotent.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message