Return-Path: Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: (qmail 84578 invoked from network); 14 Nov 2010 12:39:05 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 14 Nov 2010 12:39:05 -0000 Received: (qmail 55467 invoked by uid 500); 14 Nov 2010 12:39:37 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 55387 invoked by uid 500); 14 Nov 2010 12:39:37 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 55373 invoked by uid 99); 14 Nov 2010 12:39:36 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 14 Nov 2010 12:39:36 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 14 Nov 2010 12:39:35 +0000 Received: from thor (localhost [127.0.0.1]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id oAECdEmw017750 for ; Sun, 14 Nov 2010 12:39:14 GMT Message-ID: <20166804.68771289738354896.JavaMail.jira@thor> Date: Sun, 14 Nov 2010 07:39:14 -0500 (EST) From: "Jonathan Ellis (JIRA)" To: commits@cassandra.apache.org Subject: [jira] Commented: (CASSANDRA-1590) Pig loadfunc fails with java.io.FileNotFoundException: ...:.../job.jar!/storage-conf.xml MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931818#action_12931818 ] Jonathan Ellis commented on CASSANDRA-1590: ------------------------------------------- Did this make it into 0.6.8? > Pig loadfunc fails with java.io.FileNotFoundException: ...:.../job.jar!/storage-conf.xml > ---------------------------------------------------------------------------------------- > > Key: CASSANDRA-1590 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1590 > Project: Cassandra > Issue Type: Bug > Components: Contrib, Hadoop > Affects Versions: 0.6.4 > Environment: Ubuntu 10.04 with Hadoop from Cloudera CDH3b2 > Reporter: Tommi Virtanen > Attachments: casspig.tgz, p1590.diff > > > Trying to run the example job from contrib/pig (after fixing it to start at all in the first place; details later) results in this: > 2010-10-06 15:43:32,117 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: Store(hdfs://localhost/tmp/temp-1257182404/tmp1075428643:org.apache.pig.builtin.BinStorage) - 1-60 Operator Key: 1-60) > 2010-10-06 15:43:32,164 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer - Choosing to move algebraic foreach to combiner > 2010-10-06 15:43:32,224 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 3 > 2010-10-06 15:43:32,224 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 3 > 2010-10-06 15:43:32,302 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 > 2010-10-06 15:43:40,356 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job > 2010-10-06 15:43:40,450 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission. > 2010-10-06 15:43:40,457 [Thread-12] WARN org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. > 2010-10-06 15:43:40,950 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete > 2010-10-06 15:43:41,038 [Thread-12] INFO org.apache.cassandra.config.DatabaseDescriptor - DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap > 2010-10-06 15:43:41,211 [Thread-12] WARN org.apache.cassandra.config.DatabaseDescriptor - KeysCachedFraction is deprecated: use KeysCached instead. > 2010-10-06 15:43:41,232 [Thread-12] WARN org.apache.cassandra.config.DatabaseDescriptor - KeysCachedFraction is deprecated: use KeysCached instead. > 2010-10-06 15:43:42,305 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201010061447_0008 > 2010-10-06 15:43:42,305 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201010061447_0008 > 2010-10-06 15:44:15,025 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 33% complete > 2010-10-06 15:44:17,037 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete > 2010-10-06 15:44:17,037 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map reduce job(s) failed! > 2010-10-06 15:44:17,067 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed! > 2010-10-06 15:44:17,199 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Unable to recreate exception from backed error: Error: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml (No such file or directory) > Details at logfile: /home/tv/casspig/cassandra/contrib/pig/pig_1286405010154.log > Contents of that pig_*.log: > Backend error message > --------------------- > Error: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml (No such file or directory) > at org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:542) > at org.apache.cassandra.hadoop.ConfigHelper.getThriftPort(ConfigHelper.java:188) > at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.(ColumnFamilyRecordReader.java:118) > at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.(ColumnFamilyRecordReader.java:104) > at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.initialize(ColumnFamilyRecordReader.java:93) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(PigRecordReader.java:133) > at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > Caused by: java.io.FileNotFoundException: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml (No such file or directory) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.(FileInputStream.java:106) > at java.io.FileInputStream.(FileInputStream.java:66) > at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70) > at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161) > at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653) > at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186) > at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772) > at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737) > at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119) > at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235) > at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284) > at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:208) > at org.apache.cassandra.utils.XMLUtils.(XMLUtils.java:43) > at org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:167) > ... 9 more > Pig Stack Trace > --------------- > ERROR 2997: Unable to recreate exception from backed error: Error: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml (No such file or directory) > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias topnames > at org.apache.pig.PigServer.openIterator(PigServer.java:607) > at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:544) > at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241) > at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162) > at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138) > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75) > at org.apache.pig.Main.main(Main.java:380) > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2997: Unable to recreate exception from backed error: Error: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml (No such file or directory) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:231) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:175) > at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:270) > at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:308) > at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1007) > at org.apache.pig.PigServer.store(PigServer.java:697) > at org.apache.pig.PigServer.openIterator(PigServer.java:590) > ... 6 more > ================================================================================ > I'm attaching a tarball with everything needed to reproduce this, see the run script there. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.