hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nanda kumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-399) Handle pipeline discovery on SCM restart.
Date Mon, 17 Sep 2018 11:16:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617386#comment-16617386

Nanda kumar commented on HDDS-399:

Thanks [~msingh] for updating the patch and for the explanation on mini cluster SCM restart
changes, it makes sense.

In {{PipelineSelector}} as part of {{getReplicationPipeline}} call, we create a new pipeline
whenever possible. We make {{manager.createPipeline(replicationFactor, replicationType)}}
call to create the pipeline, which also adds the pipeline to {{PipelineManager#ActivePipeline}}
list. At this point in time, the pipeline is in ALLOCATED state. We should move the pipeline
to active allocation path only when the container is created and moved to OPEN state.
With existing code, if the pipeline initialization/updating its state fails, we are not removing
the container from the active list. This will cause an issue if that particular pipeline is
picked up for container allocation.

> Handle pipeline discovery on SCM restart.
> -----------------------------------------
>                 Key: HDDS-399
>                 URL: https://issues.apache.org/jira/browse/HDDS-399
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: SCM
>    Affects Versions: 0.2.1
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>            Priority: Blocker
>             Fix For: 0.2.1
>         Attachments: HDDS-399.001.patch, HDDS-399.002.patch, HDDS-399.003.patch, HDDS-399.004.patch,
HDDS-399.005.patch, HDDS-399.006.patch, HDDS-399.007.patch, HDDS-399.008.patch, HDDS-399.009.patch
> On SCM restart, as part on node registration, SCM should find out the list on open pipeline
on the node. Once all the nodes of the pipeline have reported back, they should be added as
active pipelines for further allocations.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message