hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5021) Add an addDirectoryToClassPath method DistributedCache
Date Fri, 22 Feb 2013 18:36:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584523#comment-13584523
] 

Alejandro Abdelnur commented on MAPREDUCE-5021:
-----------------------------------------------

Chris, I'm not sure I follow how the FileUtil method would help here. As I see this JIRA the
idea is to be able to do:

{code}
  DistributedCache.addFileToClasspath(new Path("/user/tucu/lib/*"), jobConf);
{code}

Instead doing:

{code}
  FileStatus[] files = fileStatus.listStatus(new Path("/user/tucu/lib"));
  for (FileStatus[] file : files) {
    if (file.getPath().toString().endsWith(".jar") {
      DistributedCache.addFileToClasspath(file.getPath(), jobConf);
    }
  }
{code}

In other words, do the above logic within DistributedCache.

                
> Add an addDirectoryToClassPath method DistributedCache
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-5021
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5021
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client, distributed-cache
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sandy Ryza
>
> As adding a directory of jars to the class path is a common use for the distributed cache
it would be easier on API consumers if they were able to call a method that would add all
the the files in a directory for them. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message