hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4974) Analyze and add annotations to Namenode and Datanode protocol methods to enable retry
Date Mon, 15 Jul 2013 19:40:49 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13708864#comment-13708864

Suresh Srinivas commented on HDFS-4974:

bq. I don't believe DatanodeProtocol#blockReceivedAndDeleted is safe to retry, due to things
like modifying the pending replication count.
Current processing, I agree is not retry safe. This is not being done using FSNamesystem.
I will move this to FSNamesystem and add retry cache support.

I still need to double check if blockReport() is really idempotent. Especially with the difference
in processing firstBlockReport and the subsequent one.

bq. ClientProtocol#metaSave opens its output file for append, so retries would cause duplication
of data in the output file.
Given the entire metasave is done with writeLock() held, will this be an issue? Currently
one could have two metasaves issued by two clients right? Given that I think it should be
okay to mark it as idempotent?

bq. Is it correct that ClientProtocol#rollEdits is idempotent, but NamenodeProtocol#rollEditLog
is non-idempotent? The two methods are nearly identical. They both call FSNamesystem#rollEditLog,
which I believe is idempotent.
I agree. We should mark NamenodeProtocol#rollEditLog as idempotent as well.

> Analyze and add annotations to Namenode and Datanode protocol methods to enable retry
> -------------------------------------------------------------------------------------
>                 Key: HDFS-4974
>                 URL: https://issues.apache.org/jira/browse/HDFS-4974
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, namenode
>            Reporter: Suresh Srinivas
> This jira is intended for:
> # Discussing current @Idempotent annotations in HDFS protocols and adding that annotation
where it is missing.
> # Discuss how retry should be enabled for non-idempotent requests.
> I will post the analysis of current methods in subsequent comment.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message