Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Wed, 26 Oct 2011 21:53:33 +0000 (UTC)
From: "Nathan Roberts (Commented) (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <313265681.20813.1319666013988.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1649431059.3517.1318904650634.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-7753) Support fadvise and
 sync_data_range in NativeIO, add ReadaheadPool class
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-7753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136451#comment-13136451 ] 

Nathan Roberts commented on HADOOP-7753:
----------------------------------------

Hi Todd. I reviewed the patch and only have a few questions/suggestions (Nothing that should hold it up):
* configure.ac, why the change to ldflags?
* Did you happen to test sync_file_range on both 32 and 64 bit architectures with files greater than 4G? I'm pretty sure it's correct as it basically does exactly what glibc does but I wasn't 100% certain.
* I was wondering if cancel should go ahead and remove the req from the workq. 
* I think it would nice to have a way to disable readahead via configuration. Maybe we could expose POOL_SIZE/MAX_POOL_SIZE and do it that way?
* Regarding the cancel race. It's not guaranteed to return EBADF because the fd is likely to get reused immediately for something else. Could be another file, a socket, pretty much anything. I tried to think of ways this could cause a problem, but couldn't come up with any. So I think the code is safe, maybe just a tweak to the comment saying you realize it could be a totally different file and that's ok.
* It would be nice if ReadaheadRequest sanity checked curPos and maxOffsetToRead as well, that way it's a little easier to guarantee the rest of the method is behaving correctly under all conditions.
* Might be nice in the maxOffsetToRead param to indicate what the "don't care" value is (Long.MAX_VALUE?)
                
> Support fadvise and sync_data_range in NativeIO, add ReadaheadPool class
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-7753
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7753
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: io, native
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-7753.txt, hadoop-7753.txt
>
>
> This JIRA adds JNI wrappers for sync_data_range and posix_fadvise. It also implements a ReadaheadPool class for future use from HDFS and MapReduce.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira