hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexis Fidalgo <alz...@gmail.com>
Subject about dfs.datanode.du.reserved
Date Sat, 11 Feb 2017 13:56:44 GMT
Hello, i’ve tried to search archives (and google) regarding this issue but had no luck. After
some changes in our mapreduce code, it takes all the available disk space on datanodes, before
this change we hade no problem at all, but since then, every few days, disks on datanodes
(we have 4, all with same configuration regarding disk, memory, processor, OS) becomes full
and we have no more mapreduce jobs completed. so i need to wipe datanodes and format namenode
and start all over again.

Reading documentation i found this configuration for hdfs-site.xml 

<property>
<name>dfs.datanode.du.reserved</name>
<value>32212254720</value>
</property>
</configuration>

Questions regarding this

1. is there any thread already on this issue to read and not to ask again about it?
2. if not 1, do i need to set up this property only on master or every slave too?
3. will this fix the problem or just avoid the disk become full but the MR jobs will fail
the same (no more space to work so we need to review our code)


thanks in advance, sorry if im asking about an already discussed issue, i just suscribed to
the list.

regards


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org


Mime
View raw message