hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1697) Document the behavior of -file option in streaming
Date Tue, 01 Jun 2010 17:53:38 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874123#action_12874123
] 

Hadoop QA commented on MAPREDUCE-1697:
--------------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12446011/patch-1697-2.txt
  against trunk revision 950021.

    +1 @author.  The patch does not contain any @author tags.

    +0 tests included.  The patch appears to be a documentation patch that doesn't require
tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    +1 core tests.  The patch passed core unit tests.

    -1 contrib tests.  The patch failed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/216/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/216/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/216/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/216/console

This message is automatically generated.

> Document the behavior of -file option in streaming
> --------------------------------------------------
>
>                 Key: MAPREDUCE-1697
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1697
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/streaming, documentation
>    Affects Versions: 0.20.1
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.21.0, 0.22.0
>
>         Attachments: patch-1697-1.txt, patch-1697-2.txt, patch-1697.txt
>
>
> The behavior of -file option in streaming is not documented anywhere.
> The behavior of -file is the following :
> 1) All the files passed through  -file option are packaged into job.jar.
> 2) If -file option is used for .class or .jar files, they are unjarred on tasktracker
and placed in ${mapred.local.dir}/taskTracker/jobcache/job_ID/jars/classes or /lib, respectively.
Symlinks to the directories classes and lib are created from the cwd of the task, . The names
of symlinks are "classes", "lib". So file names of .class or .jar files do not appear in cwd
of the task. 
> Paths to these files are automatically added to classpath. The tricky part is that hadoop
framework can pick .class or .jar using classpath, but actual mapper script cannot. If you'd
like to access these .class or .jar inside script, please do something like "java -cp lib/*;classes/*
<ClassName>". 
> 3) If -file option is used for files other than .class or .jar (e.g, .txt or .pl), these
files are unjarred into ${mapred.local.dir}/taskTracker/jobcache/job_ID/jars/. Symlinks to
these files are created from the cwd of the task. Names of these symlinks are actually file
names. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message