hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dieter Plaetinck (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3584) streaming.jar -file packaging forgets timestamps
Date Tue, 20 Dec 2011 17:34:30 GMT
streaming.jar -file packaging forgets timestamps

                 Key: MAPREDUCE-3584
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3584
             Project: Hadoop Map/Reduce
          Issue Type: Bug
    Affects Versions: 0.20.2
            Reporter: Dieter Plaetinck

When invoking "hadoop jar /usr/local/hadoop/contrib/streaming/hadoop-0.20.2-streaming.jar
-file <files>",
hadoop will package the files <files>, but it will forget about their timestamps.
After the files are unpacked in <tmp_dir>/mapred/local/taskTracker/jobcache/job_$job/jars,
all files will have the timestamps of when the files were unpacked.
The problem is that this way meaningful information is lost.
For example in my case i ship some files along with my job, and I need to compare the age
(mtime) of 2 files and rebuild one of them if it's too old,
but because of this hadoop behavior, my logic breaks.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message