hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8246) Get HDFS file name based on block pool id and block id
Date Tue, 28 Apr 2015 20:56:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14518034#comment-14518034
] 

Hadoop QA commented on HDFS-8246:
---------------------------------

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  14m 26s | Pre-patch trunk compilation is healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any @author tags.
|
| {color:green}+1{color} | tests included |   0m  0s | The patch appears to include 3 new
or modified test files. |
| {color:green}+1{color} | whitespace |   0m  0s | The patch has no lines that end in whitespace.
|
| {color:green}+1{color} | javac |   7m 29s | There were no new javac warning messages. |
| {color:green}+1{color} | javadoc |   9m 35s | There were no new javadoc warning messages.
|
| {color:green}+1{color} | release audit |   0m 22s | The applied patch does not increase
the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle |   7m 50s | The applied patch generated  10  additional
checkstyle issues. |
| {color:green}+1{color} | install |   1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 33s | The patch built with eclipse:eclipse.
|
| {color:green}+1{color} | findbugs |   4m 44s | The patch does not introduce any new Findbugs
(version 2.0.3) warnings. |
| {color:green}+1{color} | common tests |  22m 54s | Tests passed in hadoop-common. |
| {color:green}+1{color} | hdfs tests | 165m  8s | Tests passed in hadoop-hdfs. |
| | | 234m 40s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | http://issues.apache.org/jira/secure/attachment/12728038/HDFS-8246.0.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 99fe03e |
| checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/10431/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-common test log | https://builds.apache.org/job/PreCommit-HDFS-Build/10431/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/10431/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/10431/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep
3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/10431/console |


This message was automatically generated.

> Get HDFS file name based on block pool id and block id
> ------------------------------------------------------
>
>                 Key: HDFS-8246
>                 URL: https://issues.apache.org/jira/browse/HDFS-8246
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: HDFS, hdfs-client, namenode
>            Reporter: feng xu
>            Assignee: feng xu
>         Attachments: HDFS-8246.0.patch
>
>
> This feature provides HDFS shell command and C/Java API to retrieve HDFS file name based
on block pool id and block id.
> 1. The Java API in class DistributedFileSystem
> public String getFileName(String poolId, long blockId) throws IOException
> 2. The C API in hdfs.c
> char* hdfsGetFileName(hdfsFS fs, const char* poolId, int64_t blockId)
> 3. The HDFS shell command 
>  hdfs dfs [generic options] -fn <poolId> <blockId>
> This feature is useful if you have HDFS block file name in local file system and want
to  find out the related HDFS file name in HDFS name space (http://stackoverflow.com/questions/10881449/how-to-find-file-from-blockname-in-hdfs-hadoop).
 Each HDFS block file name in local file system contains both block pool id and block id,
for sample HDFS block file name /hdfs/1/hadoop/hdfs/data/current/BP-97622798-10.3.11.84-1428081035160/current/finalized/subdir0/subdir0/blk_1073741825,
 the block pool id is BP-97622798-10.3.11.84-1428081035160 and the block id is 1073741825.
The block  pool id is uniquely related to a HDFS name node/name space,  and the block id is
uniquely related to a HDFS file within a HDFS name node/name space, so the combination of
block pool id and a block id is uniquely related a HDFS file name. 
> The shell command and C/Java API do not map the block pool id to name node, so it’s
user’s responsibility to talk to the correct name node in federation environment that has
multiple name nodes. The block pool id is used by name node to check if the user is talking
with the correct name node.
> The implementation is straightforward. The client request to get HDFS file name reaches
the new method String getFileName(String poolId, long blockId) in FSNamesystem in name node
through RPC,  and the new method does the followings,
> (1)	Validate the block pool id.
> (2)	Create Block  based on the block id.
> (3)	Get BlockInfoContiguous from Block.
> (4)	Get BlockCollection from BlockInfoContiguous.
> (5)	Get file name from BlockCollection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message