hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1641) Job submission should fail if same uri is added for mapred.cache.files and mapred.cache.archives
Date Mon, 17 May 2010 03:25:43 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Amareshwari Sriramadasu updated MAPREDUCE-1641:

    Status: Open  (was: Patch Available)

bq. Should mapred.cache.libjars be checked too? 
Dick, there is no such configuration. files passed in -libjars are added mapreduce.job.classpath.files
to and mapreduce.job.cache.files. 

The patch needs to be updated to trunk. so canceling.

> Job submission should fail if same uri is added for mapred.cache.files and mapred.cache.archives
> ------------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-1641
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1641
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: distributed-cache
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Dick King
>             Fix For: 0.22.0
>         Attachments: BZ-3539321--off-0-20-101--2010-04-20.patch, duped-files-archives--off-0-20-101--2010-04-21.patch,
duped-files-archives--off-0-20-101--2010-04-23--1819.patch, mapreduce-1641--2010-04-27.patch,
> The behavior of mapred.cache.files and mapred.cache.archives is different during localization
in the following way:
> If a jar file is added to mapred.cache.files,  it will be localized under TaskTracker
under a unique path. 
> If a jar file is added to mapred.cache.archives, it will be localized under a unique
path in a directory named the jar file name, and will be unarchived under the same directory.
> If same jar file is passed for both the configurations, the behavior undefined. Thus
the job submission should fail.
> Currently, since distributed cache processes files before archives, the jar file will
be just localized and not unarchived.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message