hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yang <teddyyyy...@gmail.com>
Subject why does hadoop creates /tmp/hadoop-user/hadoop-unjar-xxxx/ dir and unjar my fat jar?
Date Fri, 24 Oct 2014 18:51:37 GMT
I just noticed that when I run a "hadoop jar
my-fat-jar-with-all-dependencies.jar" , it unjars the job jar in
/tmp/hadoop-username/hadoop-unjar-xxxx/ and extracts all the classes in

the fat jar is pretty big, so it took up a lot of space (particularly
inodes ) and ran out of quota.

I wonder why do we have to unjar these classes on the **client node** ? the
jar won't even be accessed until on the compute nodes, right?

View raw message