hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11493) Ozone: SCM: Add the ability to handle container reports
Date Fri, 14 Jul 2017 07:30:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Anu Engineer updated HDFS-11493:
--------------------------------
    Attachment: HDFS-11493-HDFS-7240.005.patch

bq. Should we reuse the existing one in ShutdownThreadsHelper.shutdownExecutorService?
Couple of issues, including the fact that it does not take a logger for us to 
log the failure of shutdown.

bq. ScmConfigKeys.java
Fixed.

bq. OzoneConfigKeys.java
Fixed.

bq. maxContainerReportThreads can be a local variable.
Fixed.

bq.  this can be pulled into a generic CollectionUtil if it does not exist in hadoop-common/hadoop-hdfs

The equivalent functions exist. But they seem to be before Java 7/8 time frame. This function
just calls into start java collection API.


bq.  conf parameter can be removed as it is not used
Fixed.

bq. adding a counter as you mentioned in the TODO is a good idea
Added the counter.

bq. Should we rename it to InProgressPool if this is for a single pool being processed?
Fixed.

bq. should we change “<“ to “>” to indicate that we have done waiting for the
maxWaitTime?
Thanks for catching this. Fixed.

bq. one UNKNOWN node in the pool can cost 100s. Should we reduce the maxTry from 1000 to 100
here?
Fixed.

bq. do we miss the size/keycount for the ContainerInfo of the ContainerReport?
Right now, we don't need that since all we are going to track is the container names and state.
We will get to this sometime later.


> Ozone: SCM:  Add the ability to handle container reports 
> ---------------------------------------------------------
>
>                 Key: HDFS-11493
>                 URL: https://issues.apache.org/jira/browse/HDFS-11493
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Anu Engineer
>            Assignee: Anu Engineer
>         Attachments: container-replication-storage.pdf, exploring-scalability-scm.pdf,
HDFS-11493-HDFS-7240.001.patch, HDFS-11493-HDFS-7240.002.patch, HDFS-11493-HDFS-7240.003.patch,
HDFS-11493-HDFS-7240.004.patch, HDFS-11493-HDFS-7240.005.patch
>
>
> Once a datanode sends the container report it is SCM's responsibility to determine if
the replication levels are acceptable. If it is not, SCM should initiate a replication request
to another datanode. This JIRA tracks how SCM  handles a container report.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message