hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11935) Provide optional native implementation of stat syscall.
Date Tue, 16 Jun 2015 18:35:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588534#comment-14588534
] 

Colin Patrick McCabe commented on HADOOP-11935:
-----------------------------------------------

bq. Yikes. \[the ls command\] is not portable at all when usernames > 8 characters are
in play. 

Actually, we don't use the {{ls}} command-- we use the {{stat}} command.  It works fine with
usernames greater than 8 characters.

{code}
keter:/home/cmccabe $ stat -Lc "%s,%F,%Y,%X,%a,%U,%G,%N" /tmp/t
708166,regular file,1434479406,1434479406,644,supercalifragalistic,users,‘/tmp/t’
{code}

Parsing is messy, and forking is slow.  I was hoping that we could move to Java7 nio.  Sadly,
it looks like due to the sticky bit, we won't be able to.  But using JNI for stat would still
be a great idea when {{libhadoop.so}} is present.

> Provide optional native implementation of stat syscall.
> -------------------------------------------------------
>
>                 Key: HADOOP-11935
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11935
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs, native
>            Reporter: Chris Nauroth
>         Attachments: HADOOP-11935-NativeIO-prelim.patch
>
>
> Currently, {{RawLocalFileSystem.DeprecatedRawLocalFileStatus#loadPermissionInfo}} is
implemented as forking an {{ls}} command and parsing the output.  This was observed to be
a bottleneck in YARN-3491.  This issue proposes an optional native implementation of a {{stat}}
syscall through JNI.  We would maintain the existing code as a fallback for systems where
the native code is not available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message