hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-12256) Ozone : handle inactive containers on DataNode
Date Wed, 13 Sep 2017 21:44:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-12256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chen Liang updated HDFS-12256:
    Attachment: HDFS-12256-HDFS-7240.001.patch

A closer look shows that, the active flag seems unnecessary since:
1. active is set to false if and only if when containerData is null. Similarly for active
= true
2. both active flag and containerData are final variable and thus set in constructor and never
Because of these 2, checking active flag is equivalent of checking containerData is null or
not. So did two things in v001 patch:
1. remove the active flag
2. since it is possible we store a null containerData in ContainerStatus, there are a couple
places that need to check if {{ContainerStatus#getContainer}} returns a null before proceeding.

> Ozone : handle inactive containers on DataNode
> ----------------------------------------------
>                 Key: HDFS-12256
>                 URL: https://issues.apache.org/jira/browse/HDFS-12256
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>              Labels: ozoneMerge, tocheck
>         Attachments: HDFS-12256-HDFS-7240.001.patch
> When a container gets created, corresponding metadata gets added to {{ContainerManagerImpl#containerMap}}.
What {{containerMap}} stores is a containerName to {{ContainerStatus}} instance map. When
datanode starts, it also loads this map from disk file metadata. As long as the containerName
is found in this map, it is considered an existing container.
> An issue we saw was that, occasionally, when the container creation on datanode fails,
the metadata of the failed container may still get added to {{containerMap}}, with active
flag set to false. But currently such containers are not being handled, containers with active=false
are just treated as normal containers. Then when someone tries to write to this container,
fails can happen.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message