hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nanda kumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-737) Introduce Incremental Container Report
Date Tue, 06 Nov 2018 09:39:03 GMT

    [ https://issues.apache.org/jira/browse/HDDS-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16676432#comment-16676432

Nanda kumar commented on HDDS-737:

[~jnp], thanks for the review.
{quote}In CloseContainerCommandHandler#handle the container state should be set to CLOSING
before making a ratis call.
This is done as part of HDDS-801.
{quote}pipelineManager is set in ContainerReportHandler but never used.
Both in ContainerReportHandler and IncrementalContainerReportHandler, pipelineManager will
be required when we handle state change. We need to remove the container from OPEN pipeline
when the container is moved to CLOSED state. For now, added TODO in both the classes. When
we handle state change, pipelineManager will be used.
{quote}Heartbeating thread can also receive interrupt when shutting down
Good catch. Updated the comment.
{quote}NewNodeHandler does nothing. Shouldn't it send command for a container report?
NewNode event is triggered by NodeManager, it has already made an entry for the registered
node in NodeStateManager. We get container report as part of register call, and that container
report will be processed by ContainerReportHandler to update the container replica state.
We currently have nothing to do when we receive a new node event from NodeManager. NewNodeHandler
is just a placeholder for now, in future, if required, we can use it.
{quote}Why is removeNode removed from NodeManager? It seems like the right place.
We currently don't remove a node from NodeManager once it is registered. We can add removeNode
logic when we implement decommissioning of a datanode. (existing removeNode logic was incomplete).


{quote}I prefer to add additional try-catch for thread.sleep and get InterruptedException.
Since we also have to handle {{InterruptedException}} when the shutdown is initiated, I feel
it is better to have try-catch for the complete code inside the while loop.

> Introduce Incremental Container Report
> --------------------------------------
>                 Key: HDDS-737
>                 URL: https://issues.apache.org/jira/browse/HDDS-737
>             Project: Hadoop Distributed Data Store
>          Issue Type: Improvement
>          Components: Ozone Datanode, SCM
>            Reporter: Nanda kumar
>            Assignee: Nanda kumar
>            Priority: Major
>         Attachments: HDDS-737.000.patch, HDDS-737.001.patch
> We will use Incremental Container Report (ICR) to immediately inform SCM when there is
some state change to the container in datanode. This will make sure that SCM is updated as
soon as the state of a container changes and doesn’t have to wait for full container report.
> *When do we send ICR?*
> * When a container replica state changes from open/closing to closed
> * When a container replica state changes from open/closing to quasi closed
> * When a container replica state changes from quasi closed to closed
> * When a container replica is deleted in datanode
> * When a container replica is copied from another datanode
> * When a container replica is discovered to be corrupted

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message