hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-10480) Add an admin command to list currently open files
Date Thu, 18 May 2017 18:46:04 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Manoj Govindassamy updated HDFS-10480:
    Attachment: HDFS-10480.03.patch

Thanks for the detailed review [~linyiqun]. Attached v03 patch with following comments addressed.
Please take a look.

bq. In TestDFSAdmin#verifyOpenFilesListing, line 676 LOG.info(out);, can we add one line like
LOG.info("Open files: ") before this?
Added a prefix string 'dfsadmin -listOpenFiles output:' before logging the command output.

bq. In TestDFSAdmin#verifyOpenFilesListing, line 677, the name openFilePath should be closedFilePath
Fixed the variable name.

bq. In TestListOpenFiles#verifyOpenFiles, line 145, "open files not listed!" should be " open
files not listed!". There is one space missing.
Added a space in the above logging.

bq.  Can you update the param name count to numFilesToCreate in method createOpenFiles(Path
parentDir, String fileNamePrefix, int count)? The same to closeFiles method, we can use numFilesToClose.
It will be better to understand.

bq. The operation count-- is missing in the following codes.
Good catch. Even if you don't have the count decremented, the loop gets over because of the
iterator traversal and thats why the test was passing. Nevertheless, the intention was to
close only count(numFilesToClose) files and not all. Fixed now.

bq. here should do verifyOpenFiles(openFiles) since we have closed some files and should check
the open files again. 
Added the verification here.

bq. what are following lines intended to do? In addition, the file name prefix is same.
In fact, the files are getting created with overwrite flag set to true. So, creating the files
again would overwrite them. Also, number of files to create is much more than the last iteration.
So, part of the files will get overwritten and part of the files will get newly created. To
avoid the confusion in the code, I changed the prefix to "open-3" for the last batch of creates.

PS: Few checkstyle findings are related to method length exceeding 150 and identation, which
are not introduced by the patch, so not fixing them.

> Add an admin command to list currently open files
> -------------------------------------------------
>                 Key: HDFS-10480
>                 URL: https://issues.apache.org/jira/browse/HDFS-10480
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Kihwal Lee
>            Assignee: Manoj Govindassamy
>         Attachments: HDFS-10480.02.patch, HDFS-10480.03.patch, HDFS-10480-trunk-1.patch,
> Currently there is no easy way to obtain the list of active leases or files being written.
It will be nice if we have an admin command to list open files and their lease holders.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message