hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: upstream jenkins build broken?
Date Tue, 10 Mar 2015 21:19:33 GMT
TestDataNodeHotSwapVolumes, TestDataNodeVolumeFailure,
TestDataNodeVolumeFailureReporting, and
TestDataNodeVolumeFailureToleration all remove executable permissions from
directories like the one Colin mentioned to simulate disk failures at data
nodes.  I reviewed the code for all of those, and they all appear to be
doing the necessary work to restore executable permissions at the end of
the test.  The only recent uncommitted patch I¹ve seen that makes changes
in these test suites is HDFS-7722.  That patch still looks fine though.  I
don¹t know if there are other uncommitted patches that changed these test
suites.

I suppose it¹s also possible that the JUnit process unexpectedly died
after removing executable permissions but before restoring them.  That
always would have been a weakness of these test suites, regardless of any
recent changes.

Chris Nauroth
Hortonworks
http://hortonworks.com/






On 3/10/15, 1:47 PM, "Aaron T. Myers" <atm@cloudera.com> wrote:

>Hey Colin,
>
>I asked Andrew Bayer, who works with Apache Infra, what's going on with
>these boxes. He took a look and concluded that some perms are being set in
>those directories by our unit tests which are precluding those files from
>getting deleted. He's going to clean up the boxes for us, but we should
>expect this to keep happening until we can fix the test in question to
>properly clean up after itself.
>
>To help narrow down which commit it was that started this, Andrew sent me
>this info:
>
>"/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-
>Build/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data3/ has
>500 perms, so I'm guessing that's the problem. Been that way since 9:32
>UTC
>on March 5th."
>
>--
>Aaron T. Myers
>Software Engineer, Cloudera
>
>On Tue, Mar 10, 2015 at 1:24 PM, Colin P. McCabe <cmccabe@apache.org>
>wrote:
>
>> Hi all,
>>
>> A very quick (and not thorough) survey shows that I can't find any
>> jenkins jobs that succeeded from the last 24 hours.  Most of them seem
>> to be failing with some variant of this message:
>>
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-clean-plugin:2.5:clean (default-clean)
>> on project hadoop-hdfs: Failed to clean project: Failed to delete
>>
>> 
>>/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/hadoop-hdfs-pr
>>oject/hadoop-hdfs/target/test/data/dfs/data/data3
>> -> [Help 1]
>>
>> Any ideas how this happened?  Bad disk, unit test setting wrong
>> permissions?
>>
>> Colin
>>


Mime
View raw message