crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-613) Fix FileTargetImplTest.testHandleOutputsMovesFilesToDestination instability
Date Fri, 29 Jul 2016 22:29:20 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Micah Whitacre updated CRUNCH-613:
----------------------------------
    Attachment: CRUNCH-613b.patch

Thanks for the patch.  I tweaked the test a little more to mostly cut out the use of Guava
(b/c we still want to get rid of it) and to cut down on the number of constants in that test.

> Fix FileTargetImplTest.testHandleOutputsMovesFilesToDestination instability
> ---------------------------------------------------------------------------
>
>                 Key: CRUNCH-613
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-613
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>            Reporter: Clément MATHIEU
>            Assignee: Micah Whitacre
>         Attachments: CRUNCH-613.patch, CRUNCH-613b.patch
>
>
> {{FileTargetImplTest.testHandleOutputsMovesFilesToDestination}} consistently fails on
my machine because it relies on the erroneous assumption that {{handleOutputs}} will preserve
part indexes in file names.
> If you are lucky the {{part-m-00000}} file created by the MR job will be copied to the
destination folder with the exact same name. However it just... luck. Final index only depends
on  iterator order which can vary from machine to machine (OS, FS, JDK etc.)
> This test should be rewritten to not rely on a such erroneous assumption. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message