hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12282) Ozone: DeleteKey-4: Block delete via HB between SCM and DN
Date Mon, 21 Aug 2017 23:40:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136022#comment-16136022

Anu Engineer commented on HDFS-12282:

[~cheersyang] Thanks for posting an early patch. It looks quite good. I have some minor comments.

* nit: {{DeleteBlocksCommand#staledBlocks}} could you please rename to {{DeleteBlocksCommand#blocksTobeDeleted}}
* nit: {{DeleteBlocksCommandHandler#handle}} please replace "met" with "found".
* nit:  // Send ACK back to SCM as long as meta updated
This is correct since you are writing this info to the DB. With Raft, I presume this will
be written to the replicated log. Can you file a JIRA which says deleted blocks needs to written
to Raft log also.

*  {{handleHeartbeat}} I would propose this code in handleHeartbeat runs in its own thread.
That is there is no real-time requirement that if we have a block to delete we have respond
in the very next HB. I am just worried about the time it takes to do an I/O in the HB handling
path. If you are worried that all commands are picked up and send out and you need to queue
only one or 5 delete command at a given time, we can make changes to command queue. In other
words, I think we should throttle this independent of HBs.  

Let us chat about this sometime in slack if you are free.

> Ozone: DeleteKey-4: Block delete via HB between SCM and DN
> ----------------------------------------------------------
>                 Key: HDFS-12282
>                 URL: https://issues.apache.org/jira/browse/HDFS-12282
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, ozone, scm
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>         Attachments: Block delete via HB between SCM and DN.png, HDFS-12282-HDFS-7240.001.patch,
> This is the task 3 in the design doc, implements the SCM to datanode interactions. Including
> # SCM sends block deletion message via HB to datanode
> # datanode changes block state to deleting when processes the HB response
> # datanode sends deletion ACKs back to SCM
> # SCM handles ACKs and removes blocks in DB

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message