ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adaryl \"Bob\" Wakefield, MBA" <adaryl.wakefi...@hotmail.com>
Subject Re: HDFS disk usage
Date Wed, 04 Nov 2015 23:02:40 GMT
I’ve been advised to place the data in a directory that is in /home. I attempted to do that
an got an error that said dfs.datanode.data.dir can’t be set to anything in the /home directory.

Adaryl "Bob" Wakefield, MBA
Principal
Mass Street Analytics, LLC
913.938.6685
www.linkedin.com/in/bobwakefieldmba
Twitter: @BobLovesData

From: Adaryl "Bob" Wakefield, MBA 
Sent: Wednesday, November 04, 2015 2:18 PM
To: user@ambari.apache.org 
Subject: Re: HDFS disk usage

I dug into this a little bit more. Apparently the default is 50GB.

On my datanodes, I ran df –hl. This is the output
      Filesystem Size Used Avail Use% Mounted on 
      /dev/mapper/centos-root 50G 12G 39G 23% / 
      devtmpfs 16G 0 16G 0% /dev 
      tmpfs 16G 0 16G 0% /dev/shm 
      tmpfs 16G 1.4G 15G 9% /run 
      tmpfs 16G 0 16G 0% /sys/fs/cgroup 
      /dev/sda2 494M 123M 372M 25% /boot 
      /dev/mapper/centos-home 2.7T 33M 2.7T 1% /home 


If I’m reading this right it HAS to be mounted correctly because no other mount points have
enough space correct?


Adaryl "Bob" Wakefield, MBA
Principal
Mass Street Analytics, LLC
913.938.6685
www.linkedin.com/in/bobwakefieldmba
Twitter: @BobLovesData

From: Adaryl "Bob" Wakefield, MBA 
Sent: Tuesday, November 03, 2015 10:32 AM
To: user@ambari.apache.org 
Subject: Re: HDFS disk usage

How would I go about doing that?

Adaryl "Bob" Wakefield, MBA
Principal
Mass Street Analytics, LLC
913.938.6685
www.linkedin.com/in/bobwakefieldmba
Twitter: @BobLovesData

From: Olivier Renault 
Sent: Tuesday, November 03, 2015 3:49 AM
To: user@ambari.apache.org ; user@ambari.apache.org 
Subject: Re: HDFS disk usage

Could you double check that your datanodes are using the correct mount point? 

Thanks,
Olivier
------
Olivier Renault
Solution Engineer
Mobile: +44 7500 933 036






On Tue, Nov 3, 2015 at 1:45 AM -0800, "Adaryl "Bob" Wakefield, MBA" <adaryl.wakefield@hotmail.com>
wrote:


Why is there such a large discrepancy between what is reported and my actual disk size?

B.

From: Olivier Renault 
Sent: Tuesday, November 03, 2015 3:01 AM
To: user@ambari.apache.org ; user@ambari.apache.org 
Subject: Re: HDFS disk usage

It reports the space available for HDFS. 

Thanks,
Olivier





On Mon, Nov 2, 2015 at 11:57 PM -0800, "Adaryl "Bob" Wakefield, MBA" <adaryl.wakefield@hotmail.com>
wrote:


On the dashboard, what exactly is HDFS disk usage reporting? The numbers I’m seeing are
WAY less than the total disk space on my cluster.
B.
Mime
View raw message