hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-181) CloseContainer should commit all pending open Keys on a datanode
Date Tue, 24 Jul 2018 18:15:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554616#comment-16554616
] 

Tsz Wo Nicholas Sze commented on HDDS-181:
------------------------------------------

- OpenContainerBlockMap should not be synchronized for a better performance. 
- There is a memory leak in removeContainer(..) -- it sets the entry to null instead of removing
it.
- addChunkToMap may add the same chunk twice.  See the comments below.
{code}
      keyDataSet.putIfAbsent(blockID.getLocalID(), getKeyData(info, blockID)); // (1) when
id is absent, it puts
      keyDataSet.computeIfPresent(blockID.getLocalID(), (key, value) -> { // (2) now, the
id is present, it adds again.
        value.addChunk(info);
        return value;
      });
{code}


> CloseContainer should commit all pending open Keys on a datanode
> ----------------------------------------------------------------
>
>                 Key: HDDS-181
>                 URL: https://issues.apache.org/jira/browse/HDDS-181
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: Ozone Datanode
>            Reporter: Shashikant Banerjee
>            Assignee: Shashikant Banerjee
>            Priority: Major
>             Fix For: 0.2.1
>
>         Attachments: HDDS-181.01.patch, HDDS-181.02.patch, HDDS-181.03.patch, HDDS-181.04.patch,
HDDS-181.05.patch, HDDS-181.06.patch, HDDS-181.07.patch
>
>
> A close container command arrives in the Datanode by the SCM heartBeat response.It will
then be queued up over the ratis pipeline. Once the command execution starts inside the Datanode,
it will mark the container in CLOSING State. All the pending open keys for the container now
will be committed followed by the transition of the container state from CLOSING to CLOSED.
For achieving this, all the open keys for a container need to be tracked.
> This Jira aims to address this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message