Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 14 Sep 2017 03:52:00 +0000 (UTC)
From: "Weiwei Yang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13021744.1479473973000.108641.1505361120421@Atlassian.JIRA>
In-Reply-To: <JIRA.13021744.1479473973000@Atlassian.JIRA>
References: <JIRA.13021744.1479473973000@Atlassian.JIRA> <JIRA.13021744.1479473973246@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-11156) Add new op GETFILEBLOCKLOCATIONS to
 WebHDFS REST API
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 14 Sep 2017 03:52:06 -0000


    [ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165687#comment-16165687 ] 

Weiwei Yang commented on HDFS-11156:
------------------------------------

Hi [~shahrs87]

Thanks for sharing your concern, I think you are raising up a good point. Current approach was trying to maintain compatibility as much as possible. We don't want to modify any public APIs because it will cause a lot of problems on upgrade/integrate paths. Ideally we could implement following API in {{ClientProtocol}}

{code}
public BlockLocation[] getBlockLocations(String src,
  long start, long length) throws IOException;
{code}

so that can be exposed via {{NamenodeRpcServer}} for {{NamenodeWebHdfsMethods}} to call. However because of the compatibility concerns I am not sure if this is worth to comparing to current path. It did a bit overhead on creating a dfs client, it is more compatible. 

Thanks

> Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API
> ----------------------------------------------------
>
>                 Key: HDFS-11156
>                 URL: https://issues.apache.org/jira/browse/HDFS-11156
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 2.7.3
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>             Fix For: 2.9.0, 3.0.0-alpha2
>
>         Attachments: BlockLocationProperties_JSON_Schema.jpg, BlockLocations_JSON_Schema.jpg, FileStatuses_JSON_Schema.jpg, HDFS-11156.01.patch, HDFS-11156.02.patch, HDFS-11156.03.patch, HDFS-11156.04.patch, HDFS-11156.05.patch, HDFS-11156.06.patch, HDFS-11156.07.patch, HDFS-11156.08.patch, HDFS-11156.09.patch, HDFS-11156.10.patch, HDFS-11156.11.patch, HDFS-11156.12.patch, HDFS-11156.13.patch, HDFS-11156.14.patch, HDFS-11156.15.patch, HDFS-11156.16.patch, HDFS-11156-branch-2.01.patch, Output_JSON_format_v10.jpg, SampleResponse_JSON.jpg
>
>
> Following webhdfs REST API
> {code}
> http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GET_BLOCK_LOCATIONS&offset=0&length=1
> {code}
> will get a response like
> {code}
> {
>   "LocatedBlocks" : {
>     "fileLength" : 1073741824,
>     "isLastBlockComplete" : true,
>     "isUnderConstruction" : false,
>     "lastLocatedBlock" : { ... },
>     "locatedBlocks" : [ {...} ]
>   }
> }
> {code}
> This represents for *o.a.h.h.p.LocatedBlocks*. However according to *FileSystem* API, 
> {code}
> public BlockLocation[] getFileBlockLocations(Path p, long start, long len)
> {code}
> clients would expect an array of BlockLocation. This mismatch should be fixed. Marked as Incompatible change as this will change the output of the GET_BLOCK_LOCATIONS API.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org