hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6678) Allow ShuffleHandler readahead without drop-behind
Date Mon, 09 May 2016 20:55:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277018#comment-15277018

Nathan Roberts commented on MAPREDUCE-6678:

bq. I recognize that it's difficult to produce a unit test for the patch. Would it be possible
for you to post a very brief justification of that?
The approach I took was to test it manually because it's invasive to determine whether or
not the OS is actually doing the readahead from java. Rather than create a fragile test, I
opted to use strace to verify the fadvise(WILL_NEED) occurs when the configured readahead
size is >0, and does not occur when 0; regardless of whether mapreduce.shuffle.manage.os.cache
is enabled.

> Allow ShuffleHandler readahead without drop-behind
> --------------------------------------------------
>                 Key: MAPREDUCE-6678
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6678
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 3.0.0, 2.7.2
>            Reporter: Nathan Roberts
>            Assignee: Nathan Roberts
>         Attachments: YARN-4964.001.patch
> Currently mapreduce.shuffle.manage.os.cache enables/disables both readahead (POSIX_FADV_WILLNEED)
and drop-behind (POSIX_FADV_DONTNEED) logic within the ShuffleHandler.
> It would be beneficial if these were separately configurable. 
> - Running without readahead can lead to significant seek storms caused by large numbers
of sendfiles() competing with one another.
> - However, running with drop-behind can also lead to seek storms because there are cases
where the server can successfully write the shuffle bytes to the network, BUT the client doesn't
want the bytes right now (MergeManager wants to WAIT is an example) so it ignores them and
asks for them again a bit later. This causes repeated reads of the same data from disk.
> I'll attach a simple patch that enables/disables readahead based on mapreduce.shuffle.readahead.bytes==0,
leaving mapreduce.shuffle.manage.os.cache controlling only the drop-behind.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message