hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christian Kunz (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-2231) ShellCommand, in particular 'df -k', sometimes hang
Date Mon, 19 Nov 2007 22:26:43 GMT
ShellCommand, in particular 'df -k', sometimes hang
---------------------------------------------------

                 Key: HADOOP-2231
                 URL: https://issues.apache.org/jira/browse/HADOOP-2231
             Project: Hadoop
          Issue Type: Bug
          Components: fs
    Affects Versions: 0.15.1
            Reporter: Christian Kunz


We noticed that some pipes applications writing to dfs using libhdfs have about 6% chance
of hanging when executing 'df -k' to find out whether there is enough space available on the
local filesystem before opening a file for write.

Why not using File.getFreeSpace() or File.GetUsableSpace()?

The call stack is:
Exception in thread "main" java.io.IOException
         at org.apache.hadoop.fs.ShellCommand.runCommand
(ShellCommand.java:52)
         at org.apache.hadoop.fs.ShellCommand.run(ShellCommand.java:42)
         at org.apache.hadoop.fs.DF.getAvailable(DF.java:72)
         at org.apache.hadoop.fs.LocalDirAllocator
$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:264)
         at org.apache.hadoop.fs.LocalDirAllocator
$AllocatorPerContext.createTmpFileForWrite(LocalDirAllocator.java:294)
         at
org.apache.hadoop.fs.LocalDirAllocator.createTmpFileForWrite
(LocalDirAllocator.java:155)
         at org.apache.hadoop.dfs.DFSClient
$DFSOutputStream.newBackupFile(DFSClient.java:1470)
         at org.apache.hadoop.dfs.DFSClient
$DFSOutputStream.openBackupStream(DFSClient.java:1437)
         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.writeChunk
(DFSClient.java:1579)
         at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk
(FSOutputSummer.java:140)
         at org.apache.hadoop.fs.FSOutputSummer.write1
(FSOutputSummer.java:100)
         at org.apache.hadoop.fs.FSOutputSummer.write
(FSOutputSummer.java:86)
         at org.apache.hadoop.fs.FSDataOutputStream
$PositionCache.write(FSDataOutputStream.java:39)
         at java.io.DataOutputStream.write(DataOutputStream.java:90)
         at java.io.FilterOutputStream.write(FilterOutputStream.java:80)



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message