incubator-mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <ma...@eecs.berkeley.edu>
Subject Re: spark+mesos: increasing protocol buffer message size limit
Date Mon, 11 Jun 2012 06:10:51 GMT
Are you using Mesos framework messages by any chance? It might be even better to avoid that,
because they add some overhead regardless of the protobuf size limit due to a lot of copies
happening in the stack.

Matei

On Jun 9, 2012, at 12:40 AM, Arpan Ghosh wrote:

> Hi,
> 
> I am running a spark job on a mesos cluster and would like to increase the size limit
on the protocol buffer messages. I understand that I have to call SetTotalBytesLimit() but
I wasn't sure of where the CodedInputStream was being created while running a spark job on
mesos.
> 
> 
> Thanks
> 
> Arpan
> 
> 


Mime
View raw message