hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7753) Support fadvise and sync_data_range in NativeIO, add ReadaheadPool class
Date Wed, 26 Oct 2011 21:53:33 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136451#comment-13136451

Nathan Roberts commented on HADOOP-7753:

Hi Todd. I reviewed the patch and only have a few questions/suggestions (Nothing that should
hold it up):
* configure.ac, why the change to ldflags?
* Did you happen to test sync_file_range on both 32 and 64 bit architectures with files greater
than 4G? I'm pretty sure it's correct as it basically does exactly what glibc does but I wasn't
100% certain.
* I was wondering if cancel should go ahead and remove the req from the workq. 
* I think it would nice to have a way to disable readahead via configuration. Maybe we could
expose POOL_SIZE/MAX_POOL_SIZE and do it that way?
* Regarding the cancel race. It's not guaranteed to return EBADF because the fd is likely
to get reused immediately for something else. Could be another file, a socket, pretty much
anything. I tried to think of ways this could cause a problem, but couldn't come up with any.
So I think the code is safe, maybe just a tweak to the comment saying you realize it could
be a totally different file and that's ok.
* It would be nice if ReadaheadRequest sanity checked curPos and maxOffsetToRead as well,
that way it's a little easier to guarantee the rest of the method is behaving correctly under
all conditions.
* Might be nice in the maxOffsetToRead param to indicate what the "don't care" value is (Long.MAX_VALUE?)
> Support fadvise and sync_data_range in NativeIO, add ReadaheadPool class
> ------------------------------------------------------------------------
>                 Key: HADOOP-7753
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7753
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: io, native
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-7753.txt, hadoop-7753.txt
> This JIRA adds JNI wrappers for sync_data_range and posix_fadvise. It also implements
a ReadaheadPool class for future use from HDFS and MapReduce.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message