spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <pwend...@gmail.com>
Subject Re: Problem with akka.frameSize
Date Thu, 20 Feb 2014 23:40:37 GMT
Thanks for this bug report... we'll look into this!


On Thu, Feb 20, 2014 at 8:39 AM, Guillaume Pitel <guillaume.pitel@exensa.com
> wrote:

>  Jira ticket created https://spark-project.atlassian.net/browse/SPARK-1112
>
> Guillaume
>
> Hi,
>
> I've sent a few emails to the user mailing list, but since I believe this
> is bug, I think it's time to talk to the developpers.
>
> So here is what happens: since we've migrated from 0.8.1 to 0.9, whatever
> the value of spark.akka.frameSize I set, the Executors lock when a collect
> tries to send more than 10MB of data to the driver.
>
> Since 10MB is the spark default, I suspect it could be related to
> something in the configuration. We still use System.setProperty to set the
> frameSize.
>
> As a workaround, setting the frameSize back to 10 seems to work.
>
> Guillaume
>
>
> --
>    [image: eXenSa]
>  *Guillaume PITEL, Président*
> +33(0)6 25 48 86 80
>
> eXenSa S.A.S. <http://www.exensa.com/>
>  41, rue Périer - 92120 Montrouge - FRANCE
> Tel +33(0)1 84 16 36 77 / Fax +33(0)9 72 28 37 05
>

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message