Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <505069441.1249321994813.JavaMail.jira@brutus>
Date: Mon, 3 Aug 2009 10:53:14 -0700 (PDT)
From: "Jay Booth (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-516) Low Latency distributed reads
In-Reply-To: <1395375878.1249073654783.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738455#action_12738455 ] 

Jay Booth commented on HDFS-516:
--------------------------------

I have some obligations this week but will hopefully get back to this over the weekend and put together some benchmarks.  I'm thinking I'll benchmark binary search over 20GB and 100GB sequence files (could be a semi-replacement for mapfile in certain circumstances?) and lucene search using nutch's FsDirectory implementation.  I should have something up by the 10th.

> Low Latency distributed reads
> -----------------------------
>
>                 Key: HDFS-516
>                 URL: https://issues.apache.org/jira/browse/HDFS-516
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Jay Booth
>            Priority: Minor
>         Attachments: radfs.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I created a method for low latency random reads using NIO on the server side and simulated OS paging with LRU caching and lookahead on the client side.  Some applications could include lucene searching (term->doc and doc->offset mappings are likely to be in local cache, thus much faster than nutch's current FsDirectory impl and binary search through record files (bytes at 1/2, 1/4, 1/8 marks are likely to be cached)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.