hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7294) FileUtil uses wrong stat command for FreeBSD
Date Sat, 18 Feb 2012 21:47:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211116#comment-13211116
] 

Steve Loughran commented on HADOOP-7294:
----------------------------------------

I've just committed a potentially conflict MAC specific patch to this file, HADOOP-7680, which
hard codes in {{/usr/bin/stat}} as the path for stat.

Vitalli, can we be confident that this path will always be the location of stat in BSD unix?

It may be safer to have an explicit BSD case that is separate from the rest. 

We'll also need to retain the OS_TYPE_MAC enum for compatibility. There's no use of it in
hadoop common/hdfs/mapreduce other than the {{HardLink}} file, but that doesn't mean others
don't use it.

The options then
# Add a new OS enum for BSD, give it its own command (I prefer this)
# Add a new OS enum for BSD, share the action with the MAC option
# when BSD is found, just say its OS_TYPE_MAC even though it is the other way around

I prefer the first
                
> FileUtil uses wrong stat command for FreeBSD
> --------------------------------------------
>
>                 Key: HADOOP-7294
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7294
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.21.0
>         Environment: FreeBSD 8.0-STABLE
>            Reporter: Vitalii Tymchyshyn
>         Attachments: 7294-trunk.patch, patch.diff
>
>
> I get next exception when try to use append:
> 2011-05-16 17:07:54,648 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(10.112.0.207:50010,
storageID=DS-1047171559-
> 10.112.0.207-50010-1302796304164, infoPort=50075, ipcPort=50020):DataXceiver
> java.io.IOException: Failed to get link count on file /var/data/hdfs/data/current/finalized/subdir26/subdir17/subdir55/blk_-1266943884751786595:
>  message=null; error=stat: illegal option -- c; exit value=1
>         at org.apache.hadoop.fs.FileUtil.createIOException(FileUtil.java:709)
>         at org.apache.hadoop.fs.FileUtil.access$000(FileUtil.java:42)
>         at org.apache.hadoop.fs.FileUtil$HardLink.getLinkCount(FileUtil.java:682)
>         at org.apache.hadoop.hdfs.server.datanode.ReplicaInfo.unlinkBlock(ReplicaInfo.java:215)
>         at org.apache.hadoop.hdfs.server.datanode.FSDataset.append(FSDataset.java:1116)
> It seems that FreeBSD is treated like UNIX and so calls 'stat -c%h', while FreeBSD is
much more like Mac (since they have same BSD roots):
> $ stat --help
> stat: illegal option -- -
> usage: stat [-FlLnqrsx] [-f format] [-t timefmt] [file ...]
> $ stat -f%l a_file
> 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message