hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Prakash (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11060) make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable
Date Mon, 14 Nov 2016 19:42:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664815#comment-15664815

Ravi Prakash commented on HDFS-11060:

Hi Lantao! Could you please go through https://wiki.apache.org/hadoop/HowToContribute ?
Ideally we'd like to see a patch file uploaded to the JIRA. HadoopQA would then run it through
some pre-commit tests and give a +1 (or -1) 

>From the pull request I see that you have simply made the 100 configurable. However for
pagination to work, we'd have to provide another API that takes an *offset/pageid* along with
the {{maxCorruptFileBlocksReturn}} , won't we? https://github.com/apache/hadoop/blob/ff0b99eafeda035ebe0dc82cfe689808047a8893/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java#L1281

> ---------------------------------------------------------
>                 Key: HDFS-11060
>                 URL: https://issues.apache.org/jira/browse/HDFS-11060
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs
>            Reporter: Lantao Jin
>            Priority: Minor
> Current, the easiest way to determine which blocks is missing is using NN web UI or JMX.
Unfortunately, because the DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED=100 is hard code in FSNamesystem,
only 100 missing blocks can be return by UI and JMX. Even the result of URL "https://nn:50070/fsck?listcorruptfileblocks=1&path=%2F"
is limited by this hard code value too.
> I did know FSCK can return more than 100 result but due to the security reason(with kerberos),
it is very hard to involve to costumer programs and scripts.
> So I think it should add a configurable var "maxCorruptFileBlocksReturned" to fix above
> If community also think it's worth to do, I will patch this. If not, please feel free
to tell me the reason. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message