hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer <awittena...@linkedin.com>
Subject Re: hdfs system crashes when loading files bigger than local space left
Date Fri, 16 Jul 2010 18:07:46 GMT

On Jul 16, 2010, at 3:15 AM, Vitaliy Semochkin wrote:
> That is likely way too small.
> Will setting 512Mb be better in case the whole volume size is only 190Gb?

I'd recommend at least 5gb.  I'm also assuming this same disk space isn't getting used for

> Does hadoop detect/distinct the client that uploads data from datanode and not from datanode?
> lets say I execute


> hadoop -put someFile hdfs://namenode.mycompany.com/
> from namenode.mycompany.com and from some other pc. Will it be any different for hadoop
and will hadoop orgonize data more balanced in the last case?


Again, namenode is irrelevant. Do not do put's from a datanode if you want the data to be
reasonably balanced.
View raw message