Return-Path: Delivered-To: apmail-hadoop-common-dev-archive@www.apache.org Received: (qmail 95506 invoked from network); 28 Aug 2009 18:19:40 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 28 Aug 2009 18:19:40 -0000 Received: (qmail 21730 invoked by uid 500); 28 Aug 2009 18:19:38 -0000 Delivered-To: apmail-hadoop-common-dev-archive@hadoop.apache.org Received: (qmail 21670 invoked by uid 500); 28 Aug 2009 18:19:38 -0000 Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-dev@hadoop.apache.org Received: (qmail 21634 invoked by uid 99); 28 Aug 2009 18:19:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Aug 2009 18:19:38 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [67.195.23.94] (HELO web112107.mail.gq1.yahoo.com) (67.195.23.94) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 28 Aug 2009 18:19:27 +0000 Received: (qmail 34780 invoked by uid 60001); 28 Aug 2009 18:19:05 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1251483545; bh=3WyN+Sns58aXmSiVOZQLx85cjl1T7Bbl5KSKG2DepOU=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=lS1QupSYnMjde6zmJnGwAqPYl11okHW0z02eWI/8k04duGTnfYtYqq8ZdhhxwNho81Ux9pZuCzyWJudRCU5m86BMKaRkLr90CeRvfkahpWQidCzqI+0RrO8pqjDEWrauH5Hodt1yA+T0QNpSZNVgXkN+R+YhOxTYTGKWEipBORg= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=uGwtLC0eMZdWJ2reAJLH1uuoYOAPEJxuwjUllBf6yOv+cqCigfzJewMZVygw0cDaNScE49Jyy9tAaXkowLR5BxBMaUMOp58vg497AUao7+wImzcPv5ygm9HcSxlOHB+CGslA8hgd9xDP5p08ZZ5pAYzvIhUcHIPD64pdI0w63Cw=; Message-ID: <764438.34770.qm@web112107.mail.gq1.yahoo.com> X-YMail-OSG: 3AxM6BgVM1luwmvWWYz.1kva45WeR3Qj5BsLuFHuwiZje0EucftqJz_iIgypYbOLyuBj4bb8dISCVktlEdImbxDwj7RTCFJz.QXnOXLCoNGwHEIOISn8qOAw9k7ZmHhOjo4Wn0GUNefBY2KU0FYDHQ77JMhrBcdubkFIn_DNKs_BAOLlmOXx.GGwok9YxvbsW7vJMXZA4KM2BDaE0LIl5Jtkw0y77cS8kgobPq.GAid1oD0AXq.6wV7RPDOBBn5KQJS6LZx.PCQe19Fep0dcrwweJDZepi74oJ7QZq6eyQQ50M3GGUA4DBPNYetZ Received: from [64.172.17.3] by web112107.mail.gq1.yahoo.com via HTTP; Fri, 28 Aug 2009 11:19:05 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.2 Date: Fri, 28 Aug 2009 11:19:05 -0700 (PDT) From: Steve Gao Subject: [Help] Why "java.util.zip.ZipOutputStream" need to use /tmp? To: common-user@hadoop.apache.org Cc: common-dev@hadoop.apache.org In-Reply-To: <6395.68829.qm@web112107.mail.gq1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="0-621872753-1251483545=:34770" X-Virus-Checked: Checked by ClamAV on apache.org --0-621872753-1251483545=:34770 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable would someone give us a hint? Thanks. Why "java.util.zip.ZipOutputStream" need to use /tmp? The hadoop version is 0.18.3 . Recently we got "out of space" issue. It's f= rom "java.util.zip.ZipOutputStream". We found that /tmp is full and after cleaning /tmp the problem is solved. However why hadoop needs to use /tmp? We had already configured hadoop tmp = to a local disk in: hadoop-site.xml =A0 hadoop.tmp.dir =A0 ... some large local disk ... Could it because java.util.zip.ZipOutputStream uses /tmp even if we configu= red hadoop.tmp.dir to a large local disk? The error log is here FYI: java.io.IOException: No space left on device=A0=A0=A0=A0=A0=A0=A0 =A0 at java.io.FileOutputStream.write(Native Method)=A0=A0=A0=A0=A0=A0 =A0 =A0at java.util.zip.ZipOutputStream.writeInt(ZipOutputStream.java:445)=A0= =A0=A0=A0=A0=A0=A0 =A0 at java.util.zip.ZipOutputStream.writeEXT(ZipOutputStream.java:362)=A0=A0= =A0=A0=A0=A0=A0 =A0 at java.util.zip.ZipOutputStream.closeEntry(ZipOutputStream.java:220)=A0=A0= =A0=A0=A0=A0=A0 =A0 at java.util.zip.ZipOutputStream.finish(ZipOutputStream.java:301)=A0=A0=A0= =A0=A0=A0=A0 =A0 at java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:146)= =A0=A0=A0=A0=A0=A0=A0 =A0 at java.util.zip.ZipOutputStream.close(ZipOutputStream.java:321)=A0=A0=A0= =A0=A0=A0=A0 =A0 at org.apache.hadoop.streaming.JarBuilder.merge(JarBuilder.java:79)=A0=A0= =A0=A0=A0=A0=A0 =A0 at org.apache.hadoop.streaming.StreamJob.packageJobJar(StreamJob.java:628)= =A0=A0=A0=A0=A0=A0=A0 =A0 at org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:843)=A0= =A0=A0=A0=A0=A0=A0 =A0 at org.apache.hadoop.streaming.StreamJob.go(StreamJob.java:110)=A0=A0=A0=A0= =A0=A0=A0 =A0 at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:33= )=A0=A0=A0=A0=A0=A0=A0 =A0 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)=A0=A0=A0=A0= =A0=A0=A0 =A0 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav= a:39)=A0=A0=A0=A0=A0=A0=A0 =A0 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor= Impl.java:25)=A0=A0=A0=A0=A0=A0=A0 =A0 at java.lang.reflect.Method.invoke(Method.java:597)=A0=A0=A0=A0=A0=A0=A0 = =A0 at org.apache.hadoop.util.RunJar.main(RunJar.java:155)=A0=A0=A0=A0=A0=A0=A0= =A0 at org.apache.hadoop.mapred.JobShell.run(JobShell.java:194)=A0=A0=A0=A0=A0= =A0=A0 =A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)=A0=A0=A0=A0=A0= =A0=A0 =A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)=A0=A0=A0=A0=A0= =A0=A0 =A0 at org.apache.hadoop.mapred.JobShell.main(JobShell.java:220) =A0 Executing Hadoop job failure =A0 =A0 =A0 =0A=0A=0A --0-621872753-1251483545=:34770--