hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nanda kumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12751) Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close
Date Thu, 07 Dec 2017 08:19:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16281489#comment-16281489
] 

Nanda kumar commented on HDFS-12751:
------------------------------------

Thanks [~vagarychen] for digging it.

bq. When {{ContainerMapping#updateContainerState}} gets called, it is always that {{ContainerMapper}}
itself will write the updated status to metadata db
True. What {{ContainerMapping#updateContainerState}} does is that, it reads ContainerInfo
from metadata db, updates the state (which is done through {{ContainerStateManager#updateContainerState}})
and the updated ContainerInfo is written to the db again. Here we only update/change the state
of the container, all the other informations is copied as it's from the old container info
which is read from db. We do not update {{allocatedBytes}} here.

bq. Looks like this is the only place {{allocatedSize}} gets updated
Not exactly. Whenever we allocate block in a container through {{BlockManagerImpl}}, a call
is made to {{ContainerStateManager#getMatchingContainer}} which calls {{ContainerInfo#allocate(size)}}
to update the container allocatedBytes. This is the value we want to persist in db during
SCM shutdown.

So, in {{ContainerStateManager#close}} we have to iterate through all the open containers
and update the {{allocatedBytes}} correspondingly in db.

bq. But {{ContainerMapper#close}} does not call {{containerStateManager.close()}} at all
Thanks for the catch, we have to call {{containerStateManager.close()}} in {{ContainerMapper#close}}
before {{containerStore.close()}}

> Ozone: SCM: update container allocated size to container db for all the open containers
in ContainerStateManager#close
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-12751
>                 URL: https://issues.apache.org/jira/browse/HDFS-12751
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Nanda kumar
>            Assignee: Chen Liang
>
> Container allocated size is maintained in memory by {{ContainerStateManager}}, this has
to be updated in container db when we shutdown SCM. {{ContainerStateManager#close}} will be
called during SCM shutdown, so updating allocated size for all the open containers should
be done here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message