hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2393) Mark appropriate methods of ClientProtocol with the idempotent annotation
Date Thu, 06 Oct 2011 04:36:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121717#comment-13121717

Aaron T. Myers commented on HDFS-2393:

I've been working on categorizing all of the 45 operations currently defined in {{ClientProtocol}}.
This is the breakdown of what I've come up with:

Read ops, therefore idempotent:

* getBlockLocations
* getPreferredBlockSize
* getListing
* getFileInfo
* getFileLinkInfo
* getStats
* getDatanodeReport
* listCorruptFileBlocks
* getContentSummary
* getLinkTarget
* getServerDefaults

Write ops that I believe should be marked idempotent:

* setReplication
* setPermission
* setOwner
* reportBadBlocks
* mkdirs (in fact, it looks like this op can never return false. It either returns true or
throws an exception.)
* renewLease
* setQuota
* fsync
* setTimes
* getDelegationToken
* renewDelegationToken

Write ops I'm unsure about:

* getAdditionalDatanode (I'm not sure this one should even be marked OperationCategory.WRITE
- it only gets the readLock)
* recoverLease
* updateBlockForPipeline

Write ops that I believe should *not* be idempotent:

* create
* append
* abandonBlock
* addBlock
* complete
* rename
* concat
* rename2
* delete
* createSymLink
* updatePipeline
* cancelDelegationToken

Ops which are more administrative, and arguably shouldn't be in ClientProtocol:

* setSafeMode
* saveNamespace
* restoreFailedStorage
* refreshNodes
* finalizeUpgrade
* distributedUpgradeProgress
* metaSave
* setBalancerBandwidth

Comments are most welcome. In particular, I'm interested in:

# Does anyone have any insight about the three operations I'm not sure about? We could play
it safe at first, and not mark them idempotent. Doing so will result in a few more client
failures during an NN fail over, but will not result in correctness issues.
# What to do about the 8 operations which seem administrative in nature?
# Did I misidentify any of the write operations which I believe to be idempotent?

Once we hash these things out a little bit, I'll go ahead and make a patch for it.
> Mark appropriate methods of ClientProtocol with the idempotent annotation
> -------------------------------------------------------------------------
>                 Key: HDFS-2393
>                 URL: https://issues.apache.org/jira/browse/HDFS-2393
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs client
>    Affects Versions: HA branch (HDFS-1623)
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>             Fix For: HA branch (HDFS-1623)
> HDFS-1973 will make the {{DFSClient}} take advantage of the annotation. This JIRA is
to identify which methods can be marked as idempotent.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message