hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Dunning (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2549) hdfs does not honor dfs.du.reserved setting
Date Tue, 08 Jan 2008 18:00:38 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556959#action_12556959
] 

Ted Dunning commented on HADOOP-2549:
-------------------------------------


This happens in 15.1 as well.  It is particularly problematic when there is one small and
one large partition available for storage.  If the smaller partition is listed first, then
it will be filled without any reference to available space and the status display will show
available space because the larger partition is still free.

Aggressive rebalancing can stave off the problem, but that is more of a band-aid than a solution.

It is also a real problem that when the disk fills up, the file system is corrupted in a way
that is very difficult to recover from.



> hdfs does not honor dfs.du.reserved setting
> -------------------------------------------
>
>                 Key: HADOOP-2549
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2549
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.14.4
>         Environment: FC Linux.
>            Reporter: Joydeep Sen Sarma
>            Priority: Critical
>
> running 0.14.4. one of our drives is smaller and is always getting disk full. i reset
the disk reservation to 1Gig - but it was filled quickly again.
> i put in some tracing in getnextvolume. the blocksize argument is 0. so every volume
(regardless of available space) qualifies. here's the trace:
> /* root disk chosen with 0 available bytes. format is <available>:<blocksize>*/
> 2008-01-08 08:08:51,918 WARN org.apache.hadoop.dfs.DataNode: Volume /var/hadoop/tmp/dfs/data/current:0:0
> /* some other disk chosen with 300G space. */
> 2008-01-08 08:09:21,974 WARN org.apache.hadoop.dfs.DataNode: Volume /mnt/d1/hdfs/current:304725631026:0
> i am going to default blocksize to something reasonable when it's zero for now.
> this is driving us nuts since our automounter starts failing when we run out of space.
so everything's broke.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message