hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4234) SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)
Date Fri, 20 Jul 2012 16:01:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419284#comment-13419284
] 

Robert Joseph Evans commented on MAPREDUCE-4234:
------------------------------------------------

Never mind, I thought that it was a regular unit test, but it is not.  This is used to validate
that sort worked as expected, and is a stand alone executable so we do want to throw it in
hadoop.tmp.dir, with the addition of a UUID, we probably want to clean up after we are done
anyways.
                
> SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file
with static name)
> --------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4234
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4234
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: examples
>    Affects Versions: 0.23.3, trunk
>            Reporter: Randy Clayton
>            Assignee: Robert Joseph Evans
>            Priority: Minor
>         Attachments: MAPREDUCE-4234.patch
>
>
> The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator
directory using a static filename. This can result in failures due to name collisions when
the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user
simultaneously. We use this jar when testing compression codecs and after we started running
tests in parallel (four at a time to reduce overall test time) we started experiencing random
test failures due to name collisions. Creating a random or unique per thread filename may
resolve this issue. We have developed a change to introduce per use unique file names. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message