hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Supratim Deka (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDDS-1229) Concurrency issues with Background Block Delete
Date Mon, 01 Apr 2019 16:16:00 GMT

     [ https://issues.apache.org/jira/browse/HDDS-1229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Supratim Deka resolved HDDS-1229.
    Resolution: Not A Problem

This is not an issue because HDDS-1163 adopted a simple approach to resolving the concurrency
with background block delete. Every time a chunk lookup fails when checking the block DB,
the checker retries lookup for the specific block. If the block is not found in DB, it means
background delete has removed it and the missing chunk is not actually a corruption.

> Concurrency issues with Background Block Delete
> -----------------------------------------------
>                 Key: HDDS-1229
>                 URL: https://issues.apache.org/jira/browse/HDDS-1229
>             Project: Hadoop Distributed Data Store
>          Issue Type: Sub-task
>          Components: Ozone Datanode
>            Reporter: Supratim Deka
>            Priority: Major
> HDDS-1163 takes a simplistic approach to deal with concurrent block deletes on a container,
> when the metadata scanner is checking existence of chunks for each block in the Container
Block DB.
> As part of HDDS-1663 checkBlockDB() just does a retry if any inconsistency is detected
during a concurrency window. The retry is expected to succeed because the new DB iterator
will not include any of the blocks being processed by the concurrent background delete. If
retry fails, then the inconsistency is ignored expecting the next iteration of the metadata
scanner will avoid running concurrently with the same container.
> This Jira is raised to explore a more predictable (yet simple) mechanism to deal with
this concurrency.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message