hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Mitic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8411) TestStorageDirecotyFailure, TestTaskLogsTruncater, TestWebHdfsUrl and TestSecurityUtil fail on Windows
Date Wed, 23 May 2012 01:46:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281349#comment-13281349

Ivan Mitic commented on HADOOP-8411:

Thanks Daryn, answers inline.

I don't think import org.apache.tools.ant.util.FileUtils is needed?
@Daryn: Fixed in next patch, thanks

File tmpTargetFile = new File(target.getParentFile(), "original.tmp"); will cause collisions
if files are concurrently renamed in the same directory. I'd do something more like new File(target+".tmp").
@Daryn: It does not seem it can happen in this context, but it is a good change, so fixed.

Is renameAtomicWithOverride intended to handle the issue later commented "Files that are currently
in use cannot be deleted on Windows"? If so, won't the delete of the renamed target always
@Daryn: Actually not. Renaming into the existing file fails on Windows. In this case, while
truncating the log, we are trying the rename the truncated tmp log file into the actual log
(which might exist) in which case rename fails.

I'm a bit leery of adding windows-specific behavior because those of us not running on windows
may unknowingly break it. Would it be possible to commonize the behavior by trying to do the
rename, if that fails, try to do the double rename? I'd also tag the renamed target with File#deleteOnExit()
just in case an exception happens.
@Daryn: In this case, I don't think I can get around with calling rename twice and I have
to do this dance with files. Though, your concerns are valid, and are something we are also
thinking about. At this point, goal is to make everything run on Windows, and as we move forward
we'll see what can/should be improved.
@Daryn: It makes sense to add tmpTargetFile.deleteOnExit() if regular delete fails, thanks.

Is the windows conditional in TestStorageDirectoryFailure needed? It seems like it should
work for all platforms?
@Daryn: I don't think it is required. It might make sense to leave it and test this scenario,
as on Unix like systems users can go ahead and delete these files (OS is not preventing them
from doing so). 

Have you tested that the ugi username changes in TestWebHdfsUrl won't break non-windows platforms?
@Daryn: Yes I have. Also, if you take a look at SecurityUtil.replacePattern { ...fqdn.toLowerCase()
... } you can see where the mismatch comes from.
> TestStorageDirecotyFailure, TestTaskLogsTruncater, TestWebHdfsUrl and TestSecurityUtil
fail on Windows
> ------------------------------------------------------------------------------------------------------
>                 Key: HADOOP-8411
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8411
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: util
>    Affects Versions: 1.1.0
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-8411-branch-1-win.patch, HADOOP-8411-branch-1-win.patch
>   Original Estimate: 48h
>  Remaining Estimate: 48h
> Jira tracking failures from the summary.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message