cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tommi Virtanen (JIRA)" <j...@apache.org>
Subject [jira] Updated: (CASSANDRA-1590) Pig loadfunc fails with java.io.FileNotFoundException: ...:.../job.jar!/storage-conf.xml
Date Wed, 06 Oct 2010 23:06:33 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tommi Virtanen updated CASSANDRA-1590:
--------------------------------------

    Attachment: casspig.tgz

A script with some auxiliary files to reproduce said problem from scratch. Assumes a working
Hadoop installation, pseudo-distributed mode is fine.

> Pig loadfunc fails with java.io.FileNotFoundException: ...:.../job.jar!/storage-conf.xml
> ----------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-1590
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1590
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Contrib, Hadoop
>    Affects Versions: 0.6.4
>         Environment: Ubuntu 10.04 with Hadoop from Cloudera CDH3b2
>            Reporter: Tommi Virtanen
>         Attachments: casspig.tgz
>
>
> Trying to run the example job from contrib/pig (after fixing it to start at all in the
first place; details later) results in this:
> 2010-10-06 15:43:32,117 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine
- (Name: Store(hdfs://localhost/tmp/temp-1257182404/tmp1075428643:org.apache.pig.builtin.BinStorage)
- 1-60 Operator Key: 1-60)
> 2010-10-06 15:43:32,164 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer
- Choosing to move algebraic foreach to combiner
> 2010-10-06 15:43:32,224 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size before optimization: 3
> 2010-10-06 15:43:32,224 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
- MR plan size after optimization: 3
> 2010-10-06 15:43:32,302 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2010-10-06 15:43:40,356 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
- Setting up single store job
> 2010-10-06 15:43:40,450 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map-reduce job(s) waiting for submission.
> 2010-10-06 15:43:40,457 [Thread-12] WARN  org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for the same.
> 2010-10-06 15:43:40,950 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
> 2010-10-06 15:43:41,038 [Thread-12] INFO  org.apache.cassandra.config.DatabaseDescriptor
- DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
> 2010-10-06 15:43:41,211 [Thread-12] WARN  org.apache.cassandra.config.DatabaseDescriptor
- KeysCachedFraction is deprecated: use KeysCached instead.
> 2010-10-06 15:43:41,232 [Thread-12] WARN  org.apache.cassandra.config.DatabaseDescriptor
- KeysCachedFraction is deprecated: use KeysCached instead.
> 2010-10-06 15:43:42,305 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- HadoopJobId: job_201010061447_0008
> 2010-10-06 15:43:42,305 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201010061447_0008
> 2010-10-06 15:44:15,025 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 33% complete
> 2010-10-06 15:44:17,037 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
> 2010-10-06 15:44:17,037 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 1 map reduce job(s) failed!
> 2010-10-06 15:44:17,067 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Failed!
> 2010-10-06 15:44:17,199 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Unable
to recreate exception from backed error: Error: java.lang.RuntimeException: java.io.FileNotFoundException:
/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml
(No such file or directory)
> Details at logfile: /home/tv/casspig/cassandra/contrib/pig/pig_1286405010154.log
> Contents of that pig_*.log:
> Backend error message
> ---------------------
> Error: java.lang.RuntimeException: java.io.FileNotFoundException: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml
(No such file or directory)
> 	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:542)
> 	at org.apache.cassandra.hadoop.ConfigHelper.getThriftPort(ConfigHelper.java:188)
> 	at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.<init>(ColumnFamilyRecordReader.java:118)
> 	at org.apache.cassandra.hadoop.ColumnFamilyRecordReader$RowIterator.<init>(ColumnFamilyRecordReader.java:104)
> 	at org.apache.cassandra.hadoop.ColumnFamilyRecordReader.initialize(ColumnFamilyRecordReader.java:93)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(PigRecordReader.java:133)
> 	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.io.FileNotFoundException: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml
(No such file or directory)
> 	at java.io.FileInputStream.open(Native Method)
> 	at java.io.FileInputStream.<init>(FileInputStream.java:106)
> 	at java.io.FileInputStream.<init>(FileInputStream.java:66)
> 	at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
> 	at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
> 	at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
> 	at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:772)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> 	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> 	at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
> 	at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
> 	at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:208)
> 	at org.apache.cassandra.utils.XMLUtils.<init>(XMLUtils.java:43)
> 	at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:167)
> 	... 9 more
> Pig Stack Trace
> ---------------
> ERROR 2997: Unable to recreate exception from backed error: Error: java.lang.RuntimeException:
java.io.FileNotFoundException: /var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml
(No such file or directory)
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator
for alias topnames
> 	at org.apache.pig.PigServer.openIterator(PigServer.java:607)
> 	at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:544)
> 	at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241)
> 	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:162)
> 	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:138)
> 	at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:75)
> 	at org.apache.pig.Main.main(Main.java:380)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2997: Unable to
recreate exception from backed error: Error: java.lang.RuntimeException: java.io.FileNotFoundException:
/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/attempt_201010061447_0008_m_000000_0/work/file:/var/lib/hadoop-0.20/cache/hadoop/mapred/local/taskTracker/jobcache/job_201010061447_0008/jars/job.jar!/storage-conf.xml
(No such file or directory)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:231)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:175)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:270)
> 	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:308)
> 	at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1007)
> 	at org.apache.pig.PigServer.store(PigServer.java:697)
> 	at org.apache.pig.PigServer.openIterator(PigServer.java:590)
> 	... 6 more
> ================================================================================
> I'm attaching a tarball with everything needed to reproduce this, see the run script
there.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message