Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <1331533329.1253132997517.JavaMail.jira@brutus>
Date: Wed, 16 Sep 2009 13:29:57 -0700 (PDT)
From: "Raghu Angadi (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-516) Low Latency distributed reads
In-Reply-To: <1395375878.1249073654783.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756214#action_12756214 ] 

Raghu Angadi commented on HDFS-516:
-----------------------------------


When you get a change please point me to the streaming test/benchmark.

bq. After I get those, my roadmap for this is to add checksum support and better DatanodeInfo caching. User groups would come after that.

Unless you want to add checksums for better comparison,  I don't think it is every essential. You need not spend much time on getting feature parity with HDFS. 

For more users to benefit from your work, I think it is better to extract the features that are complementary to HDFS. and we can work on getting those into HDFS. 

> Low Latency distributed reads
> -----------------------------
>
>                 Key: HDFS-516
>                 URL: https://issues.apache.org/jira/browse/HDFS-516
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Jay Booth
>            Priority: Minor
>         Attachments: hdfs-516-20090912.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> I created a method for low latency random reads using NIO on the server side and simulated OS paging with LRU caching and lookahead on the client side.  Some applications could include lucene searching (term->doc and doc->offset mappings are likely to be in local cache, thus much faster than nutch's current FsDirectory impl and binary search through record files (bytes at 1/2, 1/4, 1/8 marks are likely to be cached)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.