hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3549) use JNI-based FileStatus implementation from io.nativeio.NativeIO.POSIX#getFstat instead of shell-based implementation from RawLocalFileSystem in checkLocalDir.
Date Wed, 06 May 2015 22:42:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531590#comment-14531590
] 

Vinod Kumar Vavilapalli commented on YARN-3549:
-----------------------------------------------

Can we not get RawLocalFileSystem to automatically use native fstat if the native library
is available? That way all users can simply benefit from this seamlessly.

> use JNI-based FileStatus implementation from io.nativeio.NativeIO.POSIX#getFstat instead
of shell-based implementation from RawLocalFileSystem in checkLocalDir.
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3549
>                 URL: https://issues.apache.org/jira/browse/YARN-3549
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: zhihai xu
>            Assignee: zhihai xu
>
> Use JNI-based FileStatus implementation from io.nativeio.NativeIO.POSIX#getFstat instead
of shell-based implementation from RawLocalFileSystem in checkLocalDir.
> As discussed in YARN-3491, shell-based implementation getPermission runs shell command
"ls -ld" to get permission, which take 4 or 5 ms(very slow).
> We should switch to io.nativeio.NativeIO.POSIX#getFstat as implementation in RawLocalFileSystem
to get rid of shell-based implementation for FileStatus.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message