Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 13 Sep 2017 22:24:00 +0000 (UTC)
From: "Rushabh S Shah (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13021744.1479473973000.106477.1505341440364@Atlassian.JIRA>
In-Reply-To: <JIRA.13021744.1479473973000@Atlassian.JIRA>
References: <JIRA.13021744.1479473973000@Atlassian.JIRA> <JIRA.13021744.1479473973246@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-11156) Add new op GETFILEBLOCKLOCATIONS to
 WebHDFS REST API
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 13 Sep 2017 22:24:06 -0000


    [ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165387#comment-16165387 ] 

Rushabh S Shah commented on HDFS-11156:
---------------------------------------

I have one concern in this patch.
{code:title=NamenodeWebHdfsMethods.java|borderStyle=solid}
// Some comments here
public String get()
{
    case GETFILEBLOCKLOCATIONS:
    {
      final long offsetValue = offset.getValue();
      final Long lengthValue = length.getValue();

      FileSystem fs = FileSystem.get(conf != null ?
          conf : new Configuration());
      BlockLocation[] locations = fs.getFileBlockLocations(
          new org.apache.hadoop.fs.Path(fullpath),
          offsetValue,
          lengthValue != null? lengthValue: Long.MAX_VALUE);
      final String js = JsonUtil.toJsonString("BlockLocations",
          JsonUtil.toJsonMap(locations));
      return Response.ok(js).type(MediaType.APPLICATION_JSON).build();
    }
}
{code}
In the above code snippet, why are we creating dfsClient object which will again create an rpc to itself.
We are already in namenode. We just have to call relevant NamenodeRpcServer method.
Correct me if I am missing something.
Cc [~cheersyang] [~andrew.wang] [~liuml07] 

Also this fix introduced a bug. See more details here: HDFS-12442


> Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API
> ----------------------------------------------------
>
>                 Key: HDFS-11156
>                 URL: https://issues.apache.org/jira/browse/HDFS-11156
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 2.7.3
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>             Fix For: 2.9.0, 3.0.0-alpha2
>
>         Attachments: BlockLocationProperties_JSON_Schema.jpg, BlockLocations_JSON_Schema.jpg, FileStatuses_JSON_Schema.jpg, HDFS-11156.01.patch, HDFS-11156.02.patch, HDFS-11156.03.patch, HDFS-11156.04.patch, HDFS-11156.05.patch, HDFS-11156.06.patch, HDFS-11156.07.patch, HDFS-11156.08.patch, HDFS-11156.09.patch, HDFS-11156.10.patch, HDFS-11156.11.patch, HDFS-11156.12.patch, HDFS-11156.13.patch, HDFS-11156.14.patch, HDFS-11156.15.patch, HDFS-11156.16.patch, HDFS-11156-branch-2.01.patch, Output_JSON_format_v10.jpg, SampleResponse_JSON.jpg
>
>
> Following webhdfs REST API
> {code}
> http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GET_BLOCK_LOCATIONS&offset=0&length=1
> {code}
> will get a response like
> {code}
> {
>   "LocatedBlocks" : {
>     "fileLength" : 1073741824,
>     "isLastBlockComplete" : true,
>     "isUnderConstruction" : false,
>     "lastLocatedBlock" : { ... },
>     "locatedBlocks" : [ {...} ]
>   }
> }
> {code}
> This represents for *o.a.h.h.p.LocatedBlocks*. However according to *FileSystem* API, 
> {code}
> public BlockLocation[] getFileBlockLocations(Path p, long start, long len)
> {code}
> clients would expect an array of BlockLocation. This mismatch should be fixed. Marked as Incompatible change as this will change the output of the GET_BLOCK_LOCATIONS API.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org