mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajit Jagdale <a...@qualia-media.com>
Subject Hadoop on Mesos Memory Configuration Question
Date Fri, 02 Oct 2015 01:26:27 GMT
Hi all,

I'm new to Mesos and to using Hadoop over Mesos.  I've been trying to
determine if Mesos memory configurations are affecting the memory that I
allocate to Hadoop mappers and reducers (in Hadoop's mapped-site.xml
file).  When I set values to the mappers, something seems to interfere with
allocating that memory.

Cluster setup:
- 1 master node and 6 slave nodes
- There is no /etc/mesos-slave/resource file, so memory is configured by
Mesos.  My understanding of this is that since there are no explicit memory
settings on each slave node, Mesos is giving the asking application
(Hadoop) all of the available memory minus 1GB for running the OS.

But there still must be some mesos memory configuration somewhere, right?
Something that knows how much a slice of memory is.  I'm not sure if I know
where that is.

Any suggestions of how mesos' process of memory allocation could be
affecting how Hadoop affects memory allocation would be appreciated.

Thanks,
Ajit

Mime
View raw message