hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail
Date Fri, 07 Oct 2011 23:17:30 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123293#comment-13123293
] 

Hitesh Shah commented on MAPREDUCE-3123:
----------------------------------------

Will open a separate jira to track what should be done for escaping user input ( commands,
filenames ) in general before we use/execute them. 
                
> Symbolic links with special chars causing container/task.sh to fail
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3123
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Thomas Graves
>            Assignee: Hitesh Shah
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: MR-3123.1.patch
>
>
> the following job throws an exception when you have the special characters in it.
> hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1
-files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$&*()-_+=
-input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output
-jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
> Exception:
> 2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor:
launchContainer:
> [container-executor, hadoopuser, 1, application_1317077272567_0239,
> container_1317077272567 0239_01_000001,
> tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001,
> tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
000001/task.sh,
> tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_000001/container_1317077272567_0239_01_000001.tokens]1109221111-tests.jar:hadoop-mapreduce-p2011-09-27
> 20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit
code from container is : 2    
>                                                                                     
                      2011-09-27
> 20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception
from container-launch :  
>                                                                                     
                     
> org.apache.hadoop.util.Shell$ExitCodeException:
> /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001/task.sh:
> line 26: syntax error near unexpected token `-_+='       
> /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001/task.sh:
> line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir
test
> ink!@$&*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
>      at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)                     
                                  
>                                                                                     
                                  
>    at org.apache.hadoop.util.Shell.run(Shell.java:188)                              
                                  
>                                                                                     
                                  
>  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)       
                                  
>                                                                                     
                                 
> at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)
  
>                                                                                     
                                
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197)
 
>                                                                                     
                                
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> 2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
> 2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Processing
> container_1317077272567_0239_01_000001 of type UPDATE_DIAGNOSTICS_MSG
> 2011-09-27 20:58:48,951 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch:
> Container exited with a non-zero exit code 2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message