hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vitaliy Semochkin <vitaliy...@gmail.com>
Subject Re: hdfs system crashes when loading files bigger than local space left
Date Thu, 15 Jul 2010 08:11:12 GMT
>a) Have you set a reserved size for hdfs?
Yes. I set 128Mb as reserved size.

b) Are you loading data from the datanode?
Yes. But the datanode is running on same node as namenode (i have very small
cluster, only 5 servers and wasting one node only for namenode/jobtracker
seemed unreasonable to me)

On Wed, Jul 14, 2010 at 7:23 PM, Allen Wittenauer
<awittenauer@linkedin.com>wrote:

>
> On Jul 14, 2010, at 4:16 AM, Vitaliy Semochkin wrote:
> > Sometimes hadoop allows to load amount of data bigger than local space on
> the node.
> > Sometimes hadoop crashes and I have to reformat hdfs.
>
> a) Have you set a reserved size for hdfs?
>
> b) Are you loading data from the datanode?
>
>
>

Mime
View raw message