hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject Re: HDFS Quota Error
Date Thu, 22 May 2014 09:37:24 GMT
That means there are some or a process which are creating tons of small
files and leaving it there when the work completed.

To free up inode space you will need to delete the files.
I do not think there is any other way.

Check in your /tmp folder, how many files are there and if any process is
leaving tmp files behind.


On Thu, May 22, 2014 at 2:54 PM, Natarajan, Prabakaran 1. (NSN -
IN/Bangalore) <prabakaran.1.natarajan@nsn.com> wrote:

>  Just noted that inode is 100%.  Any better solutions to solve this?
>
>
>
> *Thanks and Regards*
>
> Prabakaran.N  aka NP
>
> nsn, Bangalore
>
> *When "I" is replaced by "We" - even Illness becomes "Wellness"*
>
>
>
>
>
> *From:* ext Natarajan, Prabakaran 1. (NSN - IN/Bangalore) [mailto:
> prabakaran.1.natarajan@nsn.com]
> *Sent:* Thursday, May 22, 2014 2:37 PM
> *To:* user@hadoop.apache.org
> *Subject:* RE: HDFS Quota Error
>
>
>
> Thanks for your reply.  But all the datanote disk has more than 50% space
> empty
>
>
>
> *Thanks and Regards*
>
> Prabakaran.N  aka NP
>
> nsn, Bangalore
>
> *When "I" is replaced by "We" - even Illness becomes "Wellness"*
>
>
>
>
>
> *From:* ext Nitin Pawar [mailto:nitinpawar432@gmail.com<nitinpawar432@gmail.com>]
>
> *Sent:* Thursday, May 22, 2014 12:56 PM
> *To:* user@hadoop.apache.org
> *Subject:* Re: HDFS Quota Error
>
>
>
> no space left on device can also mean that one of your datanode disk is
> full.
>
>
>
> Can you check disk used by each datanode.
>
>
>
> May be you will need to rebalance your replication so that some space is
> made free on this datanode.
>
>
>
> On Thu, May 22, 2014 at 12:34 PM, Natarajan, Prabakaran 1. (NSN -
> IN/Bangalore) <prabakaran.1.natarajan@nsn.com> wrote:
>
> Hi
>
>
>
> When I run a query in Hive, I get below exception.  I noticed the error
> “No space left on device”.
>
>
>
> Then I did “hadoop fs -count -q /var/local/hadoop” – which gave below
> output
>
>
>
> none             inf            none             inf           69
> 275          288034318 hdfs://nnode:54310/var/local/hadoop
>
>
>
> Why I am getting none and inf for space and remaining space quota?  Is
> this meaning is unlimited space or is there is any space left?
>
>
>
>
>
> I tried “hadoop dfsadmin -setSpaceQuota 100G /var/local/hadoop”  --  Not
> sure 100G is correct or not?  How much I need to set and how to calculate
> this?
>
>
>
> After setting 100G , I get the below output  for “hadoop fs -count -q
> /var/local/hadoop”
>
>
>
> none             inf    107374182400    104408308039           73
> 286          297777777 hdfs://nnode:54310/var/local/hadoop
>
>
>
>
>
> I have to wait to see whether 100G is going to give me an exception or
> not….
>
>
>
>
>
> --------------
>
>
>
>
>
> 2014-05-22 10:48:43,585 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:hadoop cause:java.io.FileNotFoundException:
> /var/local/hadoop/cache/mapred/local/taskTracker/hadoop/jobcache/job_201405211712_0625/jars/org/iq80/snappy/SnappyCompressor.class
> (No space left on device)
>
> 2014-05-22 10:48:43,585 WARN org.apache.hadoop.mapred.TaskTracker: Error
> initializing attempt_201405211712_0625_r_000001_2:
>
> java.io.FileNotFoundException:
> /var/local/hadoop/cache/mapred/local/taskTracker/hadoop/jobcache/job_201405211712_0625/jars/org/iq80/snappy/SnappyCompressor.class
> *(No space left on device)*
>
>         at java.io.FileOutputStream.open(Native Method)
>
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>
>         at org.apache.hadoop.util.RunJar.unJar(RunJar.java:51)
>
>         at
> org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
>
>         at
> org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
>
>         at
> org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
>
>         at
> org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:202)
>
>         at
> org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1228)
>
>         at java.security.AccessController.doPrivileged(Native Method)
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>
>         at
> org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1203)
>
>         at
> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1118)
>
>         at
> org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2430)
>
>         at java.lang.Thread.run(Thread.java:744)
>
>
>
>
>
> *Thanks and Regards*
>
> Prabakaran.N  aka NP
>
> nsn, Bangalore
>
> *When "I" is replaced by "We" - even Illness becomes "Wellness"*
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Nitin Pawar
>



-- 
Nitin Pawar

Mime
View raw message