hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Sekhon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7563) NFS gateway parseStaticMap NumberFormatException
Date Tue, 23 Dec 2014 09:37:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256787#comment-14256787

Hari Sekhon commented on HDFS-7563:

I believe it's similar but not the same since that was in a different class updateMapInternal
in 2.4.0 and iirc this static mapping functionality for the NFS gateway was added by ATM in
version 2.5.0 and I'm running 2.6.0 whereas HDFS-6361 was fixed in 2.4.1 earlier this year.

I also followed the java stack trace to the code itself as detailed above to see the line
as described above which is why I believe it's an int vs long issue on that recently added
static mapping functionality from 2.5.0.

Reproducing should be as simple as adding a 4 billion UID to /etc/nfs.map and restarting HDFS
NFS gateway.

> NFS gateway parseStaticMap NumberFormatException
> ------------------------------------------------
>                 Key: HDFS-7563
>                 URL: https://issues.apache.org/jira/browse/HDFS-7563
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: nfs
>    Affects Versions: 2.6.0
>         Environment: HDP 2.2
>            Reporter: Hari Sekhon
>            Assignee: Aaron T. Myers
>         Attachments: UID_GID_Long_HashMaps.patch
> When using the new NFS UID mapping for the HDFS NFS gateway I've discovered that my Windows
7 workstation at this bank is passing UID number 4294xxxxxx but entering this in the /etc/nfs.map
in order to remap that to a Hadoop UID prevents the NFS gateway from restarting with the error
> {code}Exception in thread "main" java.lang.NumberFormatException: For input string: "4294xxxxxx"
>         at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>         at java.lang.Integer.parseInt(Integer.java:495)
>         at java.lang.Integer.parseInt(Integer.java:527)
>         at org.apache.hadoop.security.ShellBasedIdMapping.parseStaticMap(ShellBasedIdMapping.java:318)
>         at org.apache.hadoop.security.ShellBasedIdMapping.updateMaps(ShellBasedIdMapping.java:229)
>         at org.apache.hadoop.security.ShellBasedIdMapping.<init>(ShellBasedIdMapping.java:91)
>         at org.apache.hadoop.hdfs.nfs.nfs3.RpcProgramNfs3.<init>(RpcProgramNfs3.java:176)
>         at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.<init>(Nfs3.java:45)
>         at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.startService(Nfs3.java:66)
>         at org.apache.hadoop.hdfs.nfs.nfs3.Nfs3.main(Nfs3.java:72)
> {code}
> The /etc/nfs.map file simply contains
> {code}
> uid 4294xxxxxx 1yyyy
> {code}
> It seems that the code at {code}hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedIdMapping.java{code}
> is expecting an integer at line 318 of the parseStaticMap method: {code}int remoteId
= Integer.parseInt(lineMatcher.group(2));
> int localId = Integer.parseInt(lineMatcher.group(3));{code}
> This UID does seem very high to me but it has worked successfully on a MapR-FS NFS share
and stores files created with that UID over NFS.
> The UID / GID mappings for the HDFS NFS gateway will need to be switched to using Long
to accomodate this, I've attached a patch for the parsing and UID/GID HashMaps.
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon

This message was sent by Atlassian JIRA

View raw message