hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2914) extend DistributedCache to work locally (LocalJobRunner)
Date Tue, 02 Jun 2009 10:45:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715474#action_12715474
] 

Tom White commented on HADOOP-2914:
-----------------------------------

Some initial comments:

* In DistributedCacheHandle the class doc should go before the class declaration, not at the
beginning of the file. Also need to add Apache license.
* Use an enum rather than a boolean for isArchive in CacheFile.
* We shouldn't remove public methods to DistributedCache, but rather deprecate them and remove
them in a future release. Can DistributedCache delegate to DistributedCacheManager? I like
the fact you have documented the intended audience for each public method of DistributedCache.
(This paves the way to separating the public and private interfaces in future.)
* Is there duplication between TestMRWithDistributedCache and tests that use MRCaching that
could be avoided?
* Could TestMRWithDistributedCache also test symlinking?

> extend DistributedCache to work locally (LocalJobRunner)
> --------------------------------------------------------
>
>                 Key: HADOOP-2914
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2914
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: sam rash
>            Assignee: Philip Zeyliger
>            Priority: Minor
>         Attachments: HADOOP-2914-v1-full.patch, HADOOP-2914-v1-since-4041.patch
>
>
> The DistributedCache does not work locally when using the outlined recipe at http://hadoop.apache.org/core/docs/r0.16.0/api/org/apache/hadoop/filecache/DistributedCache.html

> Ideally, LocalJobRunner would take care of populating the JobConf and copying remote
files to the local file sytem (http, assume hdfs = default fs = local fs when doing local
development.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message