incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daithi O Crualaoich (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-138) Not on right filesystem in SequentialFileNamingScheme.getSequentialFileName
Date Wed, 02 Jan 2013 19:02:13 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daithi O Crualaoich updated CRUNCH-138:
---------------------------------------

    Attachment: CRUNCH-138.patch
    
> Not on right filesystem in SequentialFileNamingScheme.getSequentialFileName
> ---------------------------------------------------------------------------
>
>                 Key: CRUNCH-138
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-138
>             Project: Crunch
>          Issue Type: Bug
>          Components: IO
>    Affects Versions: 0.4.0
>            Reporter: Daithi O Crualaoich
>         Attachments: CRUNCH-138.patch
>
>
> Exception on writing output to S3. Needs the same as in CRUNCH-47.
> 2013-01-02 16:16:43,264 INFO org.apache.hadoop.fs.s3native.NativeS3FileSystem (Thread-27):
Making dir 's3://my-bucket/out' in S3
> 2013-01-02 16:16:43,300 ERROR org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl
(Thread-27): Error in run loop
> java.lang.IllegalArgumentException: This file system object (hdfs://10.226.229.178:9000)
does not support access to the request path 's3://my-bucket/out' You possibly called FileSystem.get(conf)
when you should have called FileSystem.get(uri, conf) to obtain a file system supporting your
path.
> 	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:384)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:129)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:269)
> 	at org.apache.crunch.io.SequentialFileNamingScheme.getSequentialFileName(SequentialFileNamingScheme.java:46)
> 	at org.apache.crunch.io.SequentialFileNamingScheme.getReduceOutputName(SequentialFileNamingScheme.java:40)
> 	at org.apache.crunch.impl.mr.exec.CrunchJob.getDestFile(CrunchJob.java:101)
> 	at org.apache.crunch.impl.mr.exec.CrunchJob.handleMultiPaths(CrunchJob.java:75)
> 	at org.apache.crunch.impl.mr.exec.CrunchJob.checkRunningState(CrunchJob.java:114)
> 	at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob.checkState(CrunchControlledJob.java:278)
> 	at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.checkRunningJobs(CrunchJobControl.java:227)
> 	at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.run(CrunchJobControl.java:277)
> 	at java.lang.Thread.run(Thread.java:662)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message