hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-12751) Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close
Date Tue, 05 Dec 2017 23:23:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16279360#comment-16279360
] 

Chen Liang edited comment on HDFS-12751 at 12/5/17 11:22 PM:
-------------------------------------------------------------

I looked into the code a little bit. It appears to me that {{containerStateManager}} is a
member of {{ContainerMapping}}. When {{ContainerMapping#updateContainerState}} gets called,
it is always that {{ContainerMapper}} itself will write the updated status to metadata db
(container.db in this case), as in following lines
{code}
ContainerInfo updatedContainer = containerStateManager
          .updateContainerState(containerInfo, event);
containerStore.put(dbKey, updatedContainer.getProtobuf().toByteArray());
{code}
Looks like this is the only place {{allocatedSize}} gets updated, so I think this means the
allocated size is already being updated in container db, whenever an update happens. So we
don't need to update on close(). What do you think [~nandakumar131]? Am I missing anything?


I also noticed that when {{StorageContainerManager}} shuts down, it will call close on {{ContainerMapping}}.
But {{ContainerMapper#close}} does not call {{containerStateManager.close()}} at all. This
seems fine to me though, because {{ContainerMapping}} is doing the write to meta store by
itself, so seems okay that {{ContainerStateManager#close()}} does not do anything with current
implementation.


was (Author: vagarychen):
I looked into the code a little bit. It appears to me that {{containerStateManager}} is a
member of {{ContainerMapping}}. When {{ContainerMapping#updateContainerState}} gets called,
it is always that {{ContainerMapper}} itself will write the updated status to metadata db
(container.db in this case), as in following lines
{code}
ContainerInfo updatedContainer = containerStateManager
          .updateContainerState(containerInfo, event);
containerStore.put(dbKey, updatedContainer.getProtobuf().toByteArray());
{code}
Looks like this is the only place {{allocatedSize}} gets updated, so I think this means the
allocated size is already being updated in container db, whenever an update happens. So we
don't need to update on close(). What do you think [~nandakumar131]? Am I missing anything?


I also noticed that when {{StorageContainerManager}} shuts down, it will call close on {{ContainerMapping}}.
But {{ContainerMapper#close}} does not call {{containerStateManager.close()}} at all. This
seems fine to me though, because {{ContainerMapping}} is doing the write to meta store by
itself, so seems okay that {{ContainerStateManager#close()}} does not do anything.

> Ozone: SCM: update container allocated size to container db for all the open containers
in ContainerStateManager#close
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-12751
>                 URL: https://issues.apache.org/jira/browse/HDFS-12751
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Nanda kumar
>            Assignee: Chen Liang
>
> Container allocated size is maintained in memory by {{ContainerStateManager}}, this has
to be updated in container db when we shutdown SCM. {{ContainerStateManager#close}} will be
called during SCM shutdown, so updating allocated size for all the open containers should
be done here.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message