hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhiwen Sun <pens...@gmail.com>
Subject Re: dfs storage full on all slave machines of 6 machine hive cluster
Date Mon, 18 Mar 2013 15:10:49 GMT
The folder "/mnt/hadoop-fs/dfs/data/current/" is the main folder of
datanode in hadoop.

You can use *hadoop dfs -rmr {nouserdir} *to get more free space in HDFS.

*Don't delete file directly in OS file system.*

Zhiwen Sun



On Mon, Mar 18, 2013 at 6:48 PM, Manish Bhoge <manishbhoge@rocketmail.com>wrote:

> I think these directories belong to task tracker temporary storage. I am
> not very confident to conclude that go ahead with your clean up. So, wait
> for similar or an expert's response
>
> Sent from HTC via Rocket! excuse typo.
>
>  ------------------------------
> * From: * Chunky Gupta <chunky.gupta@vizury.com>;
> * To: * <user@hive.apache.org>;
> * Subject: * dfs storage full on all slave machines of 6 machine hive
> cluster
> * Sent: * Mon, Mar 18, 2013 10:37:39 AM
>
>   Hi,
>
> We have a 6 machine hive cluster. We are getting errors while a query is
> running and it fails. I found that on all 5 slaves storage is nearly full (
> 96%, 98%, 100%, 97%, 98% storage used) .
>
> On my slaves machines, this folder "/mnt/hadoop-fs/dfs/data/current/" is
> contributing 95% storage used. It contains folders with names "subdir0",
> "subdir1", etc and under them there are many files with name like
> "blk_-4071357924681234567" and blk_-4071357924681234567_246813.meta:, etc.
>
> I want to delete these subdir folders but I am not sure if it will not
> affects the tables which I have created.
>
> Can anyone help me and tell me what are these folders used for ?.
>
> Thanks,
> Chunky.
>

Mime
View raw message