hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13951) Precommit builds do not adequately protect against test malformed fs permissions.
Date Fri, 10 Mar 2017 18:18:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905499#comment-15905499

Sean Busbey commented on HADOOP-13951:

Updated the following jobs to do the find/chmod as a post-build step:

* PreCommit-HADOOP-Build
* PreCommit-HADOOP-OSX
* PreCommit-HDFS-Build
* PreCommit-MAPREDUCE-Build
* PreCommit-YARN-Build

Haven't updated them to do the "notice the problem and say something" part. Haven't updated
the nightly builds either, for that matter.

(N.B. that this issue still isn't assigned to me because I don't know when I'll have time
to do the above steps; anyone else with the time to work through the build changes should
feel free to step up.)

> Precommit builds do not adequately protect against test malformed fs permissions.
> ---------------------------------------------------------------------------------
>                 Key: HADOOP-13951
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13951
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: test
>            Reporter: Sean Busbey
>            Priority: Critical
> Right now this is expressed as failed Precommit-YARN-build jobs when they run on H5 /
H6 (see INFRA-13148), but the problem exists for all of the hadoop related precommit jobs.
> The issue is that we have some tests in Common (and maybe HDFS) that purposefully set
permissions within the {{target/}} directory to simulate a failure to interact with underlying
fs data. The test sets some subdirectories to have permissions such that we can no longer
delete their contents.
> Right now our precommit jobs include a step post-yetus-test-patch that traverses the
target directories and ensures that all subdirectories are modifiable:
> {code}
> find ${WORKSPACE} -name target | xargs chmod -R u+w
> {code}
> Unfortunately, if we don't get to that line (say due to an aborted build, or if the call
to yetus test-patch exceeds the job timeout), then we are left in a state where there are
still subdirectories that can't be modified (including deleted).
> Our builds also currently attempt to run a {{git clean}} at the very start of the build
after the repo is updated. If we have one of the aforementioned timeouts that leaves a can't-be-deleted
test directory, then all future builds on that machine will fail attempting to run the {{git
clean}} command.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message