Thanks,

Rahman

On Apr 14, 2014, at 2:07 PM, Saumitra <saumitra.official@gmail.com> wrote:

Hello,

Biswanath, looks like we = have confusion in calculation, 1TB would be equal to 1024GB, not 114GB.

Sandeep, I checked log directory size = as well. Log directories are hardly in few GBs, I have configured log4j pro= perties so that logs won=92t be too large.

In our = slave machines, we have 450GB disk partition for hadoop logs and DFS. Over = there logs directory is < 10GBs and rest space is occupied by DFS. 10GB = partition is for /.

Let me quote my confusion poin= t once again:

Basically I wanted to point out disc= repancy in name node status page and hadoop= dfs -dus. In my case, earlier one reports DFS usage as 1TB and= later one reports it to be 35GB. What are the factors that can cause this = difference? And why is just 35GB data causing DFS to hit its limits?
<= /blockquote>
<= /div>

I am talking about name node status= page on 50070 port. Here is the screenshot of my name node status page

<Screen Shot 2014-04-15 at 2.07.19 am.png>= ;

As I understand, 'DFS used=92 is the spac= e taken by DFS, non-DFS used is spaces taken by non-DFS data like logs or o= ther local files from users. Namenode shows that DFS used is ~1TB but hadoop dfs -dus shows it to be ~38GB.

On 14-Apr-2014, at 12:33 pm, = Sandeep Nemuri <nhsandeep6@gmail= .com> wrote:

Please check your logs directory usage.

On Mon, Apr 14, 2014 at 12:08 PM, Biswajit Nayak <biswajit.nayak= @inmobi.com> wrote:

Whats the replication factor you have? I believe it shou= ld be 3. hadoop dus shows that disk usage without replication. While name n= ode ui page gives with replication.

38gb * 3 =3D114gb ~ 1TB

~Biswa
-----oThe imp= ortant thing is not to stop questioning o-----

On Mon, Apr= 14, 2014 at 9:38 AM, Saumitra <saumitra.official@gmail.com&= gt; wrote:

Hi Biswajeet,

Non-df= s usage is ~100GB over the cluster. But still the number are nowhere near 1= TB.

Basically I wanted to point out discrepa= ncy in name node status page and hadoop dfs -dus= . In my case, earlier one reports DFS usage as 1TB and later one rep= orts it to be 35GB. What are the factors that can cause this difference? An= d why is just 35GB data causing DFS to hit its limits?

On 14-= Apr-2014, at 8:31 am, Biswajit Nayak <biswajit.nayak@inmobi.com> wrote:

Hi Saumitra,

Could you please check the non-dfs usage. They also contribute to filling u= p the disk space.
=

~Biswa
-----oThe important t= hing is not to stop questioning o-----

On Mon, Apr 14, 2014 at 1:24 AM, Saumitr= a <saumitra.official@gmail.com> wrote:

Hello,

We are runnin= g HDFS on 9-node hadoop cluster, hadoop version is 1.2.1. We are using defa= ult HDFS block size.

We have noticed that disks of= slaves are almost full. From name node=92s status page (namenode:50070), w= e could see that disks of live nodes are 90% full and DFS Used% in cluster = summary page is ~1TB.

However hadoop dfs -dus / shows that file system size is merely 38GB. 38GB number looks to be co= rrect because we keep only few Hive tables and hadoop=92s /tmp (distributed= cache and job outputs) in HDFS. All other data is cleaned up. I cross-chec= ked this from hadoop dfs -ls. Also I think that there is no internal fragment= ation because the files in our Hive tables are well-chopped in ~50MB chunks= . Here are last few lines of hadoop fsck / -files -blocks<= /font>

Status: HEALTHY
Total size: 38086441332 B
Total d= irs: 232
Total = files: 802

Total blocks (validated): <= /span>796 (avg. block size 47847288 B)
Minimally replicated= blocks: 796 (100.0 %)
Over-replicated blocks: 0 (0.0 %)

Under-replicated blocks: 6 (0.75376886 %)
Mis-replicated blocks: 0 (0.0 %)
Default replicatio= n factor: 2

Average block replication: = 3.0439699
Corrupt blocks: 0
Missing replicas: 6 (0.24762692 %)

Number of data-nodes: 9
Number of racks: = 1
FSCK ended at Sun Apr 13 19:49:23 UTC 2014 in 135 milli= seconds

My question is that why disks of s= laves are getting full even though there are only few files in DFS?

______________________________= _______________________________
The information contained in this = communication is intended solely for the use of the individual or entity to= whom it is addressed and others authorized to receive it. It may contain c= onfidential or legally privileged information. If you are not the intended = recipient you are hereby notified that any disclosure, copying, distributio= n or taking any action in reliance on the contents of this information is s= trictly prohibited and may be unlawful. If you have received this communica= tion in error, please notify us immediately by responding to this email and= then delete it from your system. The firm is neither liable for the proper= and complete transmission of the information contained in this communicati= on nor for any delay in its receipt.

_____________________________________________________________
The information contained in this commun= ication is intended solely for the use of the individual or entity to whom = it is addressed and others authorized to receive it. It may contain confide= ntial or legally privileged information. If you are not the intended recipi= ent you are hereby notified that any disclosure, copying, distribution or t= aking any action in reliance on the contents of this information is strictl= y prohibited and may be unlawful. If you have received this communication i= n error, please notify us immediately by responding to this email and then = delete it from your system. The firm is neither liable for the proper and c= omplete transmission of the information contained in this communication nor= for any delay in its receipt.

= --Regards

Sandeep Nemuri