hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-10) DFS logging in NameSystem.pendingTransfer consumes all disk space
Date Thu, 29 Dec 2011 09:19:30 GMT

     [ https://issues.apache.org/jira/browse/HDFS-10?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J resolved HDFS-10.
-------------------------

    Resolution: Won't Fix

These things help ops determine HDFS activity. If you do not wish to see them ever, you may
turn up the logging to a WARN or higher level. Its INFO by default.

Resolving as Won't Fix, as these things are useful and yet not too much info to be DEBUG-only.
                
> DFS logging in NameSystem.pendingTransfer consumes all disk space
> -----------------------------------------------------------------
>
>                 Key: HDFS-10
>                 URL: https://issues.apache.org/jira/browse/HDFS-10
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Michael Bieniosek
>
> Sometimes the namenode goes crazy.  I see this in my logs:
> 2007-04-28 02:40:46,992 INFO org.apache.hadoop.dfs.StateChange: BLOCK* NameSystem.pendingTransfer:
ask x.y.z.243:50010 to replicate blk_-9064654741761822118 to datanode(s) x.y.z.247:50010
> 2007-04-28 02:40:46,992 INFO org.apache.hadoop.dfs.StateChange: BLOCK* NameSystem.pendingTransfer:
ask x.y.z.243:50010 to replicate blk_-8996500637974689840 to datanode(s) x.y.yz.225:50010
> 2007-04-28 02:40:46,992 INFO org.apache.hadoop.dfs.StateChange: BLOCK* NameSystem.pendingTransfer:
ask x.y.z.227:50010 to replicate blk_-8870980160272831217 to datanode(s) x.y.z.244:50010
> 2007-04-28 02:40:46,992 INFO org.apache.hadoop.dfs.StateChange: BLOCK* NameSystem.pendingTransfer:
ask x.y.z.227:50010 to replicate blk_-8721101562083234290 to datanode(s) x.y.z.250:50010
> 2007-04-28 02:40:46,992 INFO org.apache.hadoop.dfs.StateChange: BLOCK* NameSystem.pendingTransfer:
ask x.y.z.250:50010 to replicate blk_-9044741671491162229 to datanode(s) x.y.z.244:50010
> There are on the order of 10k/sec until the machine runs out of disk space.
> I notice that in FSNamesystem.java, about 10 lines above this line is logged, there is
a comment:
>         //
>         // Move the block-replication into a "pending" state.
>         // The reason we use 'pending' is so we can retry
>         // replications that fail after an appropriate amount of time.
>         // (REMIND - mjc - this timer is not yet implemented.)
>         //

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message