hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject Re: HDFS Quota Error
Date Thu, 22 May 2014 07:26:13 GMT
no space left on device can also mean that one of your datanode disk is
full.

Can you check disk used by each datanode.

May be you will need to rebalance your replication so that some space is
made free on this datanode.


On Thu, May 22, 2014 at 12:34 PM, Natarajan, Prabakaran 1. (NSN -
IN/Bangalore) <prabakaran.1.natarajan@nsn.com> wrote:

>  Hi
>
> When I run a query in Hive, I get below exception.  I noticed the error
> “No space left on device”.
>
> Then I did “hadoop fs -count -q /var/local/hadoop” – which gave below
> output
>
> none             inf            none             inf           69
> 275          288034318 hdfs://nnode:54310/var/local/hadoop
>
> Why I am getting none and inf for space and remaining space quota?  Is
> this meaning is unlimited space or is there is any space left?
>
>
> I tried “hadoop dfsadmin -setSpaceQuota 100G /var/local/hadoop”  --  Not
> sure 100G is correct or not?  How much I need to set and how to calculate
> this?
>
> After setting 100G , I get the below output  for “hadoop fs -count -q
> /var/local/hadoop”
>
> none             inf    107374182400    104408308039           73
> 286          297777777 hdfs://nnode:54310/var/local/hadoop
>
>
> I have to wait to see whether 100G is going to give me an exception or
> not….
>
>
> --------------
>
>
> 2014-05-22 10:48:43,585 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:hadoop cause:java.io.FileNotFoundException:
> /var/local/hadoop/cache/mapred/local/taskTracker/hadoop/jobcache/job_201405211712_0625/jars/org/iq80/snappy/SnappyCompressor.class
> (No space left on device)
> 2014-05-22 10:48:43,585 WARN org.apache.hadoop.mapred.TaskTracker: Error
> initializing attempt_201405211712_0625_r_000001_2:
> java.io.FileNotFoundException:
> /var/local/hadoop/cache/mapred/local/taskTracker/hadoop/jobcache/job_201405211712_0625/jars/org/iq80/snappy/SnappyCompressor.class
> *(No space left on device)*
>         at java.io.FileOutputStream.open(Native Method)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>         at org.apache.hadoop.util.RunJar.unJar(RunJar.java:51)
>         at
> org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
>         at
> org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
>         at
> org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
>         at
> org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:202)
>         at
> org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1228)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at
> org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1203)
>         at
> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1118)
>         at
> org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2430)
>         at java.lang.Thread.run(Thread.java:744)
>
>
> *Thanks and Regards*
> Prabakaran.N  aka NP
> nsn, Bangalore
> *When "I" is replaced by "We" - even Illness becomes "Wellness"*
>
>
>
>
>



-- 
Nitin Pawar

Mime
View raw message