hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Starry SHI <starr...@gmail.com>
Subject Where are temp files stored?
Date Sat, 26 Sep 2009 06:34:43 GMT

I am wondering where the temp files (intermediate files) are stored. They
should be located in the hadoop.tmp.dir by default, right? why I cannot find
them in either the local file system and hdfs?

I was doing a two table join using hadoop. before the job is completed, the
intermidiate files should be stored in the tmp folder, however, I cannot
find the trace of them. Can somebody tell me how to get access to the
intermediate files in hadoop?

Another question is about the replication of the intermediate files. By
default, will the intermediate (tmp) files be written to HDFS? If yes, will
they have replicas? I am thinking if the tmp files also have replica, they
should cause a great overhead on the performance. Is there a way to specify
which files should have replica and which need not?

Looking forward to your reply!

Best regards,

/* Tomorrow is another day. So is today. */

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message