Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <6679400.1177533435678.JavaMail.jira@brutus>
Date: Wed, 25 Apr 2007 13:37:15 -0700 (PDT)
From: "Hairong Kuang (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-1296) Improve interface to
 FileSystem.getFileCacheHints
In-Reply-To: <9426052.1177524795974.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12491762 ] 

Hairong Kuang commented on HADOOP-1296:
---------------------------------------

I feel that we need either getStart() or getSize(). It seems that getSize() provides more info than getStart() because we are able to figure out the end of last block. Another question, does getFileHints still need the parameters "start" and "len"?

> Improve interface to FileSystem.getFileCacheHints
> -------------------------------------------------
>
>                 Key: HADOOP-1296
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1296
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Owen O'Malley
>         Assigned To: dhruba borthakur
>
> The FileSystem interface provides a very limited interface for finding the location of the data. The current method looks like:
> String[][] getFileCacheHints(Path file, long start, long len) throws IOException
> which returns a list of "block info" where the block info consists of a list host names. Because the hints don't include the information about where the block boundaries are, map/reduce is required to call the name node for each split. I'd propose that we fix the naming a bit and make it:
> public class BlockInfo extends Writable {
>   public long getStart();
>   public String[] getHosts();
> }
> BlockInfo[] getFileHints(Path file, long start, long len) throws IOException;
> So that map/reduce can query about the entire file and get the locations in a single call.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.