hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Zeyliger (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-476) extend DistributedCache to work locally (LocalJobRunner)
Date Mon, 24 Aug 2009 04:56:59 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Philip Zeyliger updated MAPREDUCE-476:
--------------------------------------

    Attachment: MAPREDUCE-476-v8.patch

The test failure was spurious (org.apache.hadoop.mapred.TestRecoveryManager.testRestartCount
is failing elsewhere also, and has nothing to do with this patch).

But the FindBugs errors were reasonable.  New patch fixes the three reported, and pasted in
below.

{quote}
Bad practice Warnings
Code 	Warning
DP 	org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager.makeClassLoader(ClassLoader)
creates a java.net.URLClassLoader classloader, which should be performed within a doPrivileged
block
	

Bug type DP_CREATE_CLASSLOADER_INSIDE_DO_PRIVILEGED (click for details)
In class org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager
In method org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager.makeClassLoader(ClassLoader)
In class java.net.URLClassLoader
At TaskDistributedCacheManager.java:[line 235]
RV 	org.apache.hadoop.mapred.TaskRunner.setupWorkDir(JobConf, File) ignores exceptional return
value of java.io.File.mkdir()
	

Bug type RV_RETURN_VALUE_IGNORED_BAD_PRACTICE (click for details)
In class org.apache.hadoop.mapred.TaskRunner
In method org.apache.hadoop.mapred.TaskRunner.setupWorkDir(JobConf, File)
Called method java.io.File.mkdir()
At TaskRunner.java:[line 630]
Performance Warnings
Code 	Warning
UrF 	Unread field: org.apache.hadoop.mapreduce.filecache.TaskDistributedCacheManager$CacheFile.localClassPath
{quote}

> extend DistributedCache to work locally (LocalJobRunner)
> --------------------------------------------------------
>
>                 Key: MAPREDUCE-476
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-476
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: sam rash
>            Assignee: Philip Zeyliger
>            Priority: Minor
>         Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch, HADOOP-2914-v2.patch,
HADOOP-2914-v3.patch, MAPREDUCE-476-20090814.1.txt, MAPREDUCE-476-20090818.txt, MAPREDUCE-476-v2-vs-v3.patch,
MAPREDUCE-476-v2-vs-v3.try2.patch, MAPREDUCE-476-v2-vs-v4.txt, MAPREDUCE-476-v2.patch, MAPREDUCE-476-v3.patch,
MAPREDUCE-476-v3.try2.patch, MAPREDUCE-476-v4-requires-MR711.patch, MAPREDUCE-476-v5-requires-MR711.patch,
MAPREDUCE-476-v7.patch, MAPREDUCE-476-v8.patch, MAPREDUCE-476.patch, v6-to-v7.patch
>
>
> The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html

> Ideally, LocalJobRunner would take care of populating the JobConf and copying remote
files to the local file sytem (http, assume hdfs = default fs = local fs when doing local
development.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message