hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10091) Job with a har archive as input fails on 0.23
Date Mon, 11 Nov 2013 22:36:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13819521#comment-13819521
] 

Jason Lowe commented on HADOOP-10091:
-------------------------------------

Sample backtrace from a pig job:

{noformat}
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong FS:
har://hdfs-x:x/x.har/x,
expected: hdfs://x:x
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
        at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:473)
        at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:490)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:385)
        at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
        at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1264)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:231)
        at java.lang.Thread.run(Thread.java:722)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
Caused by: java.lang.IllegalArgumentException: Wrong FS:
har://hdfs-x:x/x.har/x,
expected: hdfs://x:x
        at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:582)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:155)
        at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:76)
        at org.apache.hadoop.hdfs.DistributedFileSystem$1.<init>(DistributedFileSystem.java:416)
        at org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:409)
        at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1654)
        at org.apache.hadoop.fs.FilterFileSystem.listLocatedStatus(FilterFileSystem.java:208)
        at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:259)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
        at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:335)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
        ... 13 more
{noformat}

> Job with a har archive as input fails on 0.23
> ---------------------------------------------
>
>                 Key: HADOOP-10091
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10091
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.10
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Blocker
>
> Attempting to run a MapReduce job with a har as input fails.  Sample stacktrace to follow.
 We need to backport the fix for HADOOP-10003 to branch-0.23.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message