hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Zeyliger (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
Date Mon, 24 Aug 2009 04:56:59 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Philip Zeyliger updated MAPREDUCE-476:

    Attachment: MAPREDUCE-476-v8.patch

The test failure was spurious (org.apache.hadoop.mapred.TestRecoveryManager.testRestartCount
is failing elsewhere also, and has nothing to do with this patch).

But the FindBugs errors were reasonable.  New patch fixes the three reported, and pasted in

Bad practice Warnings
Code 	Warning
DP 	org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager.makeClassLoader(ClassLoader)
creates a java.net.URLClassLoader classloader, which should be performed within a doPrivileged

In class org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager
In method org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager.makeClassLoader(ClassLoader)
In class java.net.URLClassLoader
At TaskDistributedCacheManager.java:[line 235]
RV 	org.apache.hadoop.mapred.TaskRunner.setupWorkDir(JobConf, File) ignores exceptional return
value of java.io.File.mkdir()

Bug type RV_RETURN_VALUE_IGNORED_BAD_PRACTICE (click for details)
In class org.apache.hadoop.mapred.TaskRunner
In method org.apache.hadoop.mapred.TaskRunner.setupWorkDir(JobConf, File)
Called method java.io.File.mkdir()
At TaskRunner.java:[line 630]
Performance Warnings
Code 	Warning
UrF 	Unread field: org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager$CacheFile.localClassPath

> extend DistributedCache to work locally (LocalJobRunner)
> --------------------------------------------------------
>                 Key: MAPREDUCE-476
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: sam rash
>            Assignee: Philip Zeyliger
>            Priority: Minor
>         Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch,
HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch,
MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch,
MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch,
MAPREDUCE-476-v7.patch, MAPREDUCE-476-v8.patch, MAPREDUCE-476.patch, v6-to-v7.patch
> The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html

> Ideally, LocalJobRunner would take care of populating the JobConf and copying remote
files to the local file sytem (http, assume hdfs = default fs = local fs when doing local

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message