hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HDDS-2188) Implement LocatedFileStatus & getFileBlockLocations to provide node/localization information to Yarn/Mapreduce
Date Wed, 09 Oct 2019 18:15:00 GMT

     [ https://issues.apache.org/jira/browse/HDDS-2188?focusedWorklogId=325858&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325858
]

ASF GitHub Bot logged work on HDDS-2188:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 09/Oct/19 18:14
            Start Date: 09/Oct/19 18:14
    Worklog Time Spent: 10m 
      Work Description: avijayanhwx commented on pull request #1631: HDDS-2188 : Implement
LocatedFileStatus & getFileBlockLocations to pr…
URL: https://github.com/apache/hadoop/pull/1631
 
 
   …ovide node/localization information to Yarn/Mapreduce.
   
   >>> PRELIMINARY PATCH <<<
   
   ## What changes were proposed in this pull request?
   For applications like Hive/MapReduce to take advantage of the data locality in Ozone, Ozone
should return the location of the Ozone blocks. This is needed for better read performance
for Hadoop Applications.
   
   https://issues.apache.org/jira/browse/HDDS-2188
   
   ## How was this patch tested?
   Added a few unit tests. Working on adding more.
   Currently doing manual testing by running MR jobs.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 325858)
    Remaining Estimate: 0h
            Time Spent: 10m

> Implement LocatedFileStatus & getFileBlockLocations to provide node/localization
information to Yarn/Mapreduce
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: HDDS-2188
>                 URL: https://issues.apache.org/jira/browse/HDDS-2188
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: Ozone Filesystem
>    Affects Versions: 0.5.0
>            Reporter: Mukul Kumar Singh
>            Assignee: Aravindan Vijayan
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> For applications like Hive/MapReduce to take advantage of the data locality in Ozone,
Ozone should return the location of the Ozone blocks. This is needed for better read performance
for Hadoop Applications.
> {code}
>         if (file instanceof LocatedFileStatus) {
>           blkLocations = ((LocatedFileStatus) file).getBlockLocations();
>         } else {
>           blkLocations = fs.getFileBlockLocations(file, 0, length);
>         }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message