hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Tuttle <...@mentacapital.com>
Subject RE: difference between dus and df output
Date Thu, 31 Mar 2016 21:50:08 GMT
This is very interesting, Ted. Thank you.

We are only running HBase on hdfs.

Does this mostly empty block appending behavior make sense for HBase-only usage?

If this is, in fact, unused storage how do we get it back? 

Currently df shows 75% filled while du shows 25%.  The former is prompting us to consider
more hardware.  If in fact we are 25% we don't need to.

-----Original Message-----
From: Ted Yu [mailto:yuzhihong@gmail.com] 
Sent: Thursday, March 31, 2016 1:13 PM
To: user@hbase.apache.org
Subject: difference between dus and df output

Have you seen this thread ?

http://search-hadoop.com/m/uOzYtatlmAcqgzM

On Thu, Mar 31, 2016 at 11:58 AM, Ted Tuttle <ted@mentacapital.com> wrote:

> Hello-
>
> We are running v0.94.9 cluster.
>
> I am seeing that 'fs -dus' reports 24TB used and 'fs -df' reports 
> 74.TB used.
>
> Does anyone know why these do not reconcile? Our replication factor is 
> 2 so that is not a likely explanation.
>
> Shown below are results from my cluster (doctored to TB for ease of
> reading):
>
> bash-4.1$ hadoop fs -dus /hbase
> hdfs://host/hbase      24.5TB
>
> bash-4.1$ hadoop fs -df /hbase
> Filesystem              Size    Used    Avail   Use%
> /hbase          103.8TB 74.2TB 24.3TB  71%
>
Mime
View raw message