Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of starrysl@gmail.com designates
 209.85.223.171 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:from:date:message-id:subject:to:content-type;
        b=xMM7Lbh/vqv7XJQ0DDeGGGBoKNazklPCwyxNtf3Y7cHMooEhakykRTUjeQrbDMZAMu
         fXwWhuUO7yLfH64lAZ47koH/BB7ksjCCYVv1aKd8Cul9pGh3VAHIf0bDSjBNO20D0zQC
         fNham1gW3vW9OEDx1KrY8K1xGMySgwnMjQwCM=
MIME-Version: 1.0
From: Starry SHI <starrysl@gmail.com>
Date: Sat, 26 Sep 2009 14:34:43 +0800
Message-ID: <a5accba30909252334h782536e5v228e4abed81929a4@mail.gmail.com>
Subject: Where are temp files stored?
To: common-dev@hadoop.apache.org, common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001517741218ba1fd40474754360

--001517741218ba1fd40474754360
Content-Type: text/plain; charset=UTF-8

Hi.

I am wondering where the temp files (intermediate files) are stored. They
should be located in the hadoop.tmp.dir by default, right? why I cannot find
them in either the local file system and hdfs?

I was doing a two table join using hadoop. before the job is completed, the
intermidiate files should be stored in the tmp folder, however, I cannot
find the trace of them. Can somebody tell me how to get access to the
intermediate files in hadoop?

Another question is about the replication of the intermediate files. By
default, will the intermediate (tmp) files be written to HDFS? If yes, will
they have replicas? I am thinking if the tmp files also have replica, they
should cause a great overhead on the performance. Is there a way to specify
which files should have replica and which need not?

Looking forward to your reply!

Best regards,
Starry

/* Tomorrow is another day. So is today. */

--001517741218ba1fd40474754360--