hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chef Win2er <win2erc...@gmail.com>
Subject Trash data after upgrade from 2.7.1 to 2.7.2
Date Fri, 12 Feb 2016 06:01:01 GMT
Hi Hadoop users,

I have hadoop-2.7.1 installed on my cluster with HA, 4 data nodes and 3
journal nodes.
I upgraded it to hadoop2.7.2 a a few days ago following the steps below.

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html#Upgrade_without_Downtime

But today I realized that there's trash fold created in data node's data
directory and took a lot of space.

$ hdfs dfs -du -s -h
/

11.5 G  /

I set replication 2 so the disk usage may be 30G or 40G.
But actually it is 144GB.

$ hdfs dfsadmin -report
Configured Capacity: 422185762816 (393.19 GB)
Present Capacity: 415469745432 (386.94 GB)
DFS Remaining: 260712565164 (242.81 GB)
DFS Used: 154757180268 (144.13 GB)
DFS Used%: 37.25%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0

By 'du -h' commnand I got the result below.

......
11G     ./datanode/current/BP-606697376-<datanode
ip>-1452599640542/current/finalized/subdir0
11G     ./datanode/current/BP-606697376-<datanode
ip>-1452599640542/current/finalized
11G     ./datanode/current/BP-606697376-<datanode ip>-1452599640542/current
...
38G     ./datanode/current/BP-606697376-<datanode
ip>-1452599640542/trash/finalized/subdir0
38G     ./datanode/current/BP-606697376-<datanode
ip>-1452599640542/trash/finalized
38G     ./datanode/current/BP-606697376-<datanode ip>-1452599640542/trash
...

Could anyone help me with this?

Thanks
MA

Mime
View raw message