Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <40597BEA6F39F74CA51ABDDA45D6E97D0B3781@S2K8MNTAEXC3.mentacapital.local>
References: 
 <CALte62yVG1xD60=Yk8T+ziprRqvBBeaVMZLcnnnCdc4UDTNhvg@mail.gmail.com>
	<40597BEA6F39F74CA51ABDDA45D6E97D0B3781@S2K8MNTAEXC3.mentacapital.local>
Date: Thu, 31 Mar 2016 14:57:03 -0700
Message-ID: 
 <CALte62zs80GHpZh5xKszz2P4g0znDUYVYH74P+SxYBH8jYrR6g@mail.gmail.com>
Subject: Re: difference between dus and df output
From: Ted Yu <yuzhihong@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=94eb2c0a8c9ab006ae052f5f5711

--94eb2c0a8c9ab006ae052f5f5711
Content-Type: text/plain; charset=UTF-8

Have you performed major compaction lately ?

Are there non-expired hbase snapshots ?

Cheers

On Thu, Mar 31, 2016 at 2:50 PM, Ted Tuttle <ted@mentacapital.com> wrote:

> This is very interesting, Ted. Thank you.
>
> We are only running HBase on hdfs.
>
> Does this mostly empty block appending behavior make sense for HBase-only
> usage?
>
> If this is, in fact, unused storage how do we get it back?
>
> Currently df shows 75% filled while du shows 25%.  The former is prompting
> us to consider more hardware.  If in fact we are 25% we don't need to.
>
> -----Original Message-----
> From: Ted Yu [mailto:yuzhihong@gmail.com]
> Sent: Thursday, March 31, 2016 1:13 PM
> To: user@hbase.apache.org
> Subject: difference between dus and df output
>
> Have you seen this thread ?
>
> http://search-hadoop.com/m/uOzYtatlmAcqgzM
>
> On Thu, Mar 31, 2016 at 11:58 AM, Ted Tuttle <ted@mentacapital.com> wrote:
>
> > Hello-
> >
> > We are running v0.94.9 cluster.
> >
> > I am seeing that 'fs -dus' reports 24TB used and 'fs -df' reports
> > 74.TB used.
> >
> > Does anyone know why these do not reconcile? Our replication factor is
> > 2 so that is not a likely explanation.
> >
> > Shown below are results from my cluster (doctored to TB for ease of
> > reading):
> >
> > bash-4.1$ hadoop fs -dus /hbase
> > hdfs://host/hbase      24.5TB
> >
> > bash-4.1$ hadoop fs -df /hbase
> > Filesystem              Size    Used    Avail   Use%
> > /hbase          103.8TB 74.2TB 24.3TB  71%
> >
>

--94eb2c0a8c9ab006ae052f5f5711--