crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daithi O Crualaoich (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-138) Not on right filesystem in SequentialFileNamingScheme.getSequentialFileName
Date Wed, 02 Jan 2013 18:58:14 GMT
Daithi O Crualaoich created CRUNCH-138:
------------------------------------------

             Summary: Not on right filesystem in SequentialFileNamingScheme.getSequentialFileName
                 Key: CRUNCH-138
                 URL: https://issues.apache.org/jira/browse/CRUNCH-138
             Project: Crunch
          Issue Type: Bug
          Components: IO
    Affects Versions: 0.4.0
            Reporter: Daithi O Crualaoich


Exception on writing output to S3. Needs the same as in CRUNCH-47.

2013-01-02 16:16:43,264 INFO org.apache.hadoop.fs.s3native.NativeS3FileSystem (Thread-27):
Making dir 's3://my-bucket/out' in S3
2013-01-02 16:16:43,300 ERROR org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl
(Thread-27): Error in run loop
java.lang.IllegalArgumentException: This file system object (hdfs://10.226.229.178:9000) does
not support access to the request path 's3://my-bucket/out' You possibly called FileSystem.get(conf)
when you should have called FileSystem.get(uri, conf) to obtain a file system supporting your
path.
	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:384)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:129)
	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:269)
	at org.apache.crunch.io.SequentialFileNamingScheme.getSequentialFileName(SequentialFileNamingScheme.java:46)
	at org.apache.crunch.io.SequentialFileNamingScheme.getReduceOutputName(SequentialFileNamingScheme.java:40)
	at org.apache.crunch.impl.mr.exec.CrunchJob.getDestFile(CrunchJob.java:101)
	at org.apache.crunch.impl.mr.exec.CrunchJob.handleMultiPaths(CrunchJob.java:75)
	at org.apache.crunch.impl.mr.exec.CrunchJob.checkRunningState(CrunchJob.java:114)
	at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob.checkState(CrunchControlledJob.java:278)
	at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.checkRunningJobs(CrunchJobControl.java:227)
	at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.run(CrunchJobControl.java:277)
	at java.lang.Thread.run(Thread.java:662)


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message