hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aitor Perez Cedres <ape...@pragsis.com>
Subject Re: HDFS Quota Error
Date Thu, 22 May 2014 07:33:58 GMT

Maybe you are out of space in a local disk? That location[1] looks like 
the local dir where MR places some intermediate files. Can you check the 
output of df -h on a shell?


[1] /var/local/hadoop/cache/mapred/local

On 22/05/14 09:04, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) wrote:
> Hi
> When I run a query in Hive, I get below exception.  I noticed the 
> error "No space left on device".
> Then I did "hadoop fs -count -q /var/local/hadoop" -- which gave below 
> output
> none             inf            none inf           69          
> 275          288034318 hdfs://nnode:54310/var/local/hadoop
> Why I am getting none and inf for space and remaining space quota?  Is 
> this meaning is unlimited space or is there is any space left?
> I tried "hadoop dfsadmin -setSpaceQuota 100G /var/local/hadoop"  --  
> Not sure 100G is correct or not? How much I need to set and how to 
> calculate this?
> After setting 100G , I get the below output  for "hadoop fs -count -q 
> /var/local/hadoop"
> none             inf    107374182400 104408308039           
> 73          286          297777777 hdfs://nnode:54310/var/local/hadoop
> I have to wait to see whether 100G is going to give me an exception or 
> not....
> --------------
> 2014-05-22 10:48:43,585 ERROR 
> org.apache.hadoop.security.UserGroupInformation: 
> PriviledgedActionException as:hadoop 
> cause:java.io.FileNotFoundException: 
> /var/local/hadoop/cache/mapred/local/taskTracker/hadoop/jobcache/job_201405211712_0625/jars/org/iq80/snappy/SnappyCompressor.class

> (No space left on device)
> 2014-05-22 10:48:43,585 WARN org.apache.hadoop.mapred.TaskTracker: 
> Error initializing attempt_201405211712_0625_r_000001_2:
> java.io.FileNotFoundException: 
> /var/local/hadoop/cache/mapred/local/taskTracker/hadoop/jobcache/job_201405211712_0625/jars/org/iq80/snappy/SnappyCompressor.class

> *(No space left on device)*
>         at java.io.FileOutputStream.open(Native Method)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>         at org.apache.hadoop.util.RunJar.unJar(RunJar.java:51)
>         at 
> org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
>         at 
> org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
>         at 
> org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
>         at 
> org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:202)
>         at 
> org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1228)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         at 
> org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1203)
>         at 
> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1118)
>         at 
> org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2430)
>         at java.lang.Thread.run(Thread.java:744)
> *Thanks and Regards*
> Prabakaran.N  aka NP
> nsn, Bangalore
> */When "I" is replaced by "We" - even Illness becomes "Wellness"/*

-- 
*Aitor PĂ©rez*
/Big Data System Engineer/

Telf.: +34 917 680 490
Fax: +34 913 833 301
C/Manuel Tovar, 49-53 - 28034 Madrid - Spain

_http://www.bidoop.es_


Mime
View raw message