apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim <...@facility.supplies>
Subject RE: Application Memory Reducing
Date Mon, 15 Feb 2016 14:06:44 GMT
Tushar,

I found the yarn-site.xml, but how do I restart the yarn service?

Does ‘restart the gateway’ under Configuration do that?

Thanks,

Jim

From: Tushar Gosavi [mailto:tushar@datatorrent.com]
Sent: Monday, February 15, 2016 7:46 AM
To: users@apex.incubator.apache.org
Subject: Re: Application Memory Reducing

Hi Jim,

Can you check your yarn-site.xml for yarn.scheduler.minimum-allocation-mb, it may be set to
1G (default for 2.7 hadoop).
Reduce this value to 256MB and restart yarn service and try again running the application.

    <property>
        <name>yarn.scheduler.minimum-allocation-mb</name>
        <value>256</value>
    </property>

Regards,
-Tushar.


On Mon, Feb 15, 2016 at 7:06 PM, Jim <jim@facility.supplies<mailto:jim@facility.supplies>>
wrote:
I have successfully built my first test Apex package on the most recent Sandbox, but only
one half of it can start because of memory allocation.

From the article:

http://docs.datatorrent.com/tutorials/topnwords-c7/#managing-memory-allocation-for-containers

I have put the following into my properties.xml:

<property>
  <name>dt.application.*.attr.MASTER_MEMORY_MB</name>
  <value>1024</value>
</property>
<property>
  <name>dt.application.*.operator.*.attr.MEMORY_MB</name>
  <value>200</value>
</property>
<property>
  <name>dt.application.*.operator.*.port.*.attr.BUFFER_MEMORY_MB</name>
  <value>200</value>
</property>

Yet the memory allocated to my operators is still 1 GB each for the most part (1 exception
has 2 GB), as can be seen below:

[cid:image001.jpg@01D167C7.CE9C07E0]

I would like to get this down to 512MB per operator – what is not allowing the settings
to take?

I did see a comment where it may be the yarn configuration, but I don’t know where that
is within the Datatorrent Apex Sandbox configuration; nor do I know if there is anything else
that could be causing this?

Thanks,

Jim

Jim


jim@facility.supplies<mailto:jim@facility.supplies> (414) 760-7711
________________________________
The information contained in this communication, including any files or attachments transmitted
with it, may contain copyrighted information or information that is confidential and exempt
from disclosure under applicable laws and regulations, is intended only for the use of the
recipient(s) named above, and may be legally privileged. If the reader of this message is
not the intended recipient, you are hereby notified that any dissemination, distribution,
or copying of this communication, or any of its contents, files or attachments, is strictly
prohibited. If you have received this communication in error, please return it to the sender
immediately and delete the original message and any copy of it from your computer system.
If you have any questions concerning this message, please contact the sender.





________________________________
The information contained in this communication, including any files or attachments transmitted
with it, may contain copyrighted information or information that is confidential and exempt
from disclosure under applicable laws and regulations, is intended only for the use of the
recipient(s) named above, and may be legally privileged. If the reader of this message is
not the intended recipient, you are hereby notified that any dissemination, distribution,
or copying of this communication, or any of its contents, files or attachments, is strictly
prohibited. If you have received this communication in error, please return it to the sender
immediately and delete the original message and any copy of it from your computer system.
If you have any questions concerning this message, please contact the sender.

Jim


jim@facility.supplies (414) 760-7711
________________________________
The information contained in this communication, including any files or attachments transmitted
with it, may contain copyrighted information or information that is confidential and exempt
from disclosure under applicable laws and regulations, is intended only for the use of the
recipient(s) named above, and may be legally privileged. If the reader of this message is
not the intended recipient, you are hereby notified that any dissemination, distribution,
or copying of this communication, or any of its contents, files or attachments, is strictly
prohibited. If you have received this communication in error, please return it to the sender
immediately and delete the original message and any copy of it from your computer system.
If you have any questions concerning this message, please contact the sender.
Mime
View raw message