hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcel Mitsuto F. S." <mits...@gmail.com>
Subject Re: Trash data after upgrade from 2.7.1 to 2.7.2
Date Fri, 12 Feb 2016 10:50:36 GMT
Check if it you have the following files: *dncp_block_verification.log.curr
, dncp_block_verification.log.prev*
Usually they grow a lot and it's not reported within DFS usage statistics.



marcel mitsuto
http://about.me/djeps
[image: marcel on about.me]

<http://about.me/djeps>

On Fri, Feb 12, 2016 at 7:01 AM, Chef Win2er <win2erchef@gmail.com> wrote:

> Hi Hadoop users,
>
> I have hadoop-2.7.1 installed on my cluster with HA, 4 data nodes and 3
> journal nodes.
> I upgraded it to hadoop2.7.2 a a few days ago following the steps below.
>
>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html#Upgrade_without_Downtime
>
> But today I realized that there's trash fold created in data node's data
> directory and took a lot of space.
>
> $ hdfs dfs -du -s -h
> /
>
> 11.5 G  /
>
> I set replication 2 so the disk usage may be 30G or 40G.
> But actually it is 144GB.
>
> $ hdfs dfsadmin -report
> Configured Capacity: 422185762816 (393.19 GB)
> Present Capacity: 415469745432 (386.94 GB)
> DFS Remaining: 260712565164 (242.81 GB)
> DFS Used: 154757180268 (144.13 GB)
> DFS Used%: 37.25%
> Under replicated blocks: 0
> Blocks with corrupt replicas: 0
> Missing blocks: 0
> Missing blocks (with replication factor 1): 0
>
> By 'du -h' commnand I got the result below.
>
> ......
> 11G     ./datanode/current/BP-606697376-<datanode
> ip>-1452599640542/current/finalized/subdir0
> 11G     ./datanode/current/BP-606697376-<datanode
> ip>-1452599640542/current/finalized
> 11G     ./datanode/current/BP-606697376-<datanode ip>-1452599640542/current
> ...
> 38G     ./datanode/current/BP-606697376-<datanode
> ip>-1452599640542/trash/finalized/subdir0
> 38G     ./datanode/current/BP-606697376-<datanode
> ip>-1452599640542/trash/finalized
> 38G     ./datanode/current/BP-606697376-<datanode ip>-1452599640542/trash
> ...
>
> Could anyone help me with this?
>
> Thanks
> MA
>

Mime
View raw message