hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7714) Add support in native libs for OS buffer cache management
Date Thu, 06 Oct 2011 18:16:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122127#comment-13122127
] 

Nathan Roberts commented on HADOOP-7714:
----------------------------------------

Todd, you may already be well aware of this, but just in case...

Patterns like the one below don't usually do what one would expect, especially if the data
has to go over a wire. I believe the reason is due to the way the socket buffers inside the
kernel keep track of the data that needs to be sent. It's basically just a reference to the
page cache page. Therefore, if the data has not actually left the box when the fadvise is
called, the references are still there so the pages cannot  be invalidated. I tried this with
a small native app and a 128MB file, and sure enough everything except for the first few pages
stayed in the page cache. 

I can't immediately think of a surefire way around this. We could just call fadvise once at
close and just live with the fact that everything still buffered at the time won't be affected.
We could do what Cristina was doing and always call fadvise with offset of 0 so that we try
to invalidate pages multiple times. We could call the fadvise asynchronously after a second
or so. Delaying a bit might help us deal with hot blocks better as well. 
sendfile(4, 5, [131072000], 65536)      = 65536
sendfile(4, 5, [131137536], 65536)      = 65536
sendfile(4, 5, [131203072], 65536)      = 65536
sendfile(4, 5, [131268608], 65536)      = 65536
sendfile(4, 5, [131334144], 65536)      = 65536
sendfile(4, 5, [131399680], 65536)      = 65536
sendfile(4, 5, [131465216], 65536)      = 65536
sendfile(4, 5, [131530752], 65536)      = 65536
sendfile(4, 5, [131596288], 65536)      = 65536
sendfile(4, 5, [131661824], 65536)      = 65536
sendfile(4, 5, [131727360], 65536)      = 65536
sendfile(4, 5, [131792896], 65536)      = 65536
sendfile(4, 5, [131858432], 65536)      = 65536
sendfile(4, 5, [131923968], 65536)      = 65536
sendfile(4, 5, [131989504], 65536)      = 65536
sendfile(4, 5, [132055040], 65536)      = 65536
fadvise64(5, 131072000, 1048576, POSIX_FADV_DONTNEED) = 0

                
> Add support in native libs for OS buffer cache management
> ---------------------------------------------------------
>
>                 Key: HADOOP-7714
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7714
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: native
>    Affects Versions: 0.24.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-7714-20s-prelim.txt
>
>
> Especially in shared HBase/MR situations, management of the OS buffer cache is important.
Currently, running a big MR job will evict all of HBase's hot data from cache, causing HBase
performance to really suffer. However, caching of the MR input/output is rarely useful, since
the datasets tend to be larger than cache and not re-read often enough that the cache is used.
Having access to the native calls {{posix_fadvise}} and {{sync_data_range}} on platforms where
they are supported would allow us to do a better job of managing this cache.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message