hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "lohit vijayarenu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2027) FileSystem should provide byte ranges for file locations
Date Sat, 09 Feb 2008 09:16:07 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

lohit vijayarenu updated HADOOP-2027:
-------------------------------------

    Attachment: HADOOP-2027-1.patch

Thanks Owen. Attached patch includes
1. new API getFileBlockLocations which invokes getBlockLocations to return BlockLocation[]
2. Changes FileSplit to store host information and return when getLocations() is invoked 
3. Change FileInputFormat to one call of getFileBlockLocations and store host information
in FileSplit using new constructor

I ran the unit test and do not see failures. Will test benchmark and report the timings. 

> FileSystem should provide byte ranges for file locations
> --------------------------------------------------------
>
>                 Key: HADOOP-2027
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2027
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: fs
>            Reporter: Owen O'Malley
>            Assignee: lohit vijayarenu
>         Attachments: HADOOP-2027-1.patch
>
>
> FileSystem's getFileCacheHints should be replaced with something more useful. I'd suggest
replacing getFileCacheHints with a new method:
> {code}
> BlockLocation[] getFileLocations(Path file, long offset, long range) throws IOException;
> {code}
> and adding
> {code}
> class BlockLocation implements Writable {
>   String[] getHosts();
>   long getOffset();
>   long getLength();
> }
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message