spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: flaky tests & scaled timeouts
Date Thu, 28 May 2015 18:57:47 GMT
My understanding is that it is good to have this, but a large fraction of
flaky tests are actually also race conditions, etc.


On Thu, May 28, 2015 at 11:55 AM, Imran Rashid <irashid@cloudera.com> wrote:

> Hi,
>
> I was just fixing a problem with too short a timeout on one of the unit
> tests I added (https://issues.apache.org/jira/browse/SPARK-7919), and I
> was wondering if this is a common problem w/ a lot of our flaky tests.
>
> Its really hard to know what to set the timeouts to -- you set the timeout
> so it runs comfortably on your dev machine, but then discover its far too
> short for the build machines.  so up it some more, and then it turns out
> there are some even slower build machines, and you need to make it longer
> still ...
>
> Scalatest actually has a feature to handle this.  You can wrap all
> timeouts in scaled(...), and then you can pass in a flag with how much the
> timeouts should get scaled by.
>
>
> http://doc.scalatest.org/2.0/index.html#org.scalatest.concurrent.ScaledTimeSpans
>
> eg., I was looking at DriverSuite -- its been turned off because it was
> too flaky, though the 60 second timeout already seems super long. (
> https://issues.apache.org/jira/browse/SPARK-7415)
>
> does anybody know if timeouts are a common source of flaky tests?  Is it
> worth trying to change existing timeouts in tests to use scaled() so keep
> tests passing on slow build machines?  Does it make sense for all future
> tests to always use scaled() on timeouts?
>
> thanks,
> Imran
>

Mime
View raw message