incubator-mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arpan Ghosh <arpangh...@gmail.com>
Subject Re: spark+mesos: increasing protocol buffer message size limit
Date Mon, 11 Jun 2012 06:27:02 GMT
We are using actors. These are the task objects that get serialized and
sent over before a parallelize/foreach starts.

On Sun, Jun 10, 2012 at 11:10 PM, Matei Zaharia <matei@eecs.berkeley.edu>wrote:

> Are you using Mesos framework messages by any chance? It might be even
> better to avoid that, because they add some overhead regardless of the
> protobuf size limit due to a lot of copies happening in the stack.
>
> Matei
>
> On Jun 9, 2012, at 12:40 AM, Arpan Ghosh wrote:
>
> > Hi,
> >
> > I am running a spark job on a mesos cluster and would like to increase
> the size limit on the protocol buffer messages. I understand that I have to
> call SetTotalBytesLimit() but I wasn't sure of where the CodedInputStream
> was being created while running a spark job on mesos.
> >
> >
> > Thanks
> >
> > Arpan
> >
> >
>
>

Mime
View raw message