hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12196) Ozone: DeleteKey-2: Implement container recycling service to delete stale blocks at background
Date Sat, 29 Jul 2017 00:55:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105945#comment-16105945

Weiwei Yang commented on HDFS-12196:

Hi [~anu]

Thanks for the quick response

bq. Should we have generic background tasks Service?

That absolutely makes sense. I have already created an abstract class {{AbstractRecyclingService}}
which is a base class for all Recycling Services, so moving on there will be {{KsmRecyclingService}},
{{ScmRecyclingService}} and {{ContainerRecyclingService}}. What you are suggesting sounds
like we need a parent for all these backend services, but need to think about the base class
design to let it reusable for them. I will think about this.

bq. If we move to that architecture, then we should have a queue on to which the commands
are queued in the data node. 

That will be the work in next task. This patch only contains an independent recycling service,
scans containers db in certain interval and deletes stuff. In next task, I will create a {{BlockDeletionCommandHandler}}
which hands the block deletion command sent by SCM, that handler will just move blocks to
deleting state in containers db. This work will be done with SCM->Datanode interaction

Let me know if you have more thoughts, I can make improvements in v2 patch then.

Thank you.

> Ozone: DeleteKey-2: Implement container recycling service to delete stale blocks at background
> ----------------------------------------------------------------------------------------------
>                 Key: HDFS-12196
>                 URL: https://issues.apache.org/jira/browse/HDFS-12196
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>         Attachments: HDFS-12196-HDFS-7240.001.patch
> Implement a recycling service running on datanode to delete stale blocks.  The recycling
service scans staled blocks for each container and delete chunks and references periodically.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message