hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Supratim Deka (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDDS-1559) Include committedBytes to determine Out of Space in VolumeChoosingPolicy
Date Mon, 20 May 2019 03:31:00 GMT
Supratim Deka created HDDS-1559:

             Summary: Include committedBytes to determine Out of Space in VolumeChoosingPolicy
                 Key: HDDS-1559
                 URL: https://issues.apache.org/jira/browse/HDDS-1559
             Project: Hadoop Distributed Data Store
          Issue Type: Improvement
          Components: Ozone Datanode
            Reporter: Supratim Deka
            Assignee: Supratim Deka

This is a follow-up from HDDS-1511 and HDDS-1535

Currently  when creating a new Container, the DN invokes RoundRobinVolumeChoosingPolicy:chooseVolume().
This routine checks for (volume available space > container max size). If no eligible volume
is found, the policy throws a DiskOutOfSpaceException. This is the current behaviour.

However, the computation of available space does not take into consideration the space
that is going to be consumed by writes to existing containers which are still Open and accepting
chunk writes.

This Jira proposes to enhance the space availability check in chooseVolume by inclusion of
committed space(committedBytes in HddsVolume) in the equation.

The handling/management of the exception in Ratis will not be modified in this Jira. That
will be scoped separately as part of Datanode IO Failure handling work.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message