apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandesh Hegde <sand...@datatorrent.com>
Subject Re: Application Memory Reducing
Date Mon, 15 Feb 2016 16:41:41 GMT
Hello Jim,

You can try increasing the memory allocated to Sandbox.

Thanks
Sandesh

On Mon, Feb 15, 2016 at 8:36 AM Munagala Ramanath <ram@datatorrent.com>
wrote:

> Jim,
>
> On the sandbox, the start/stop scripts are at /sfw/hadoop/hadoop-2.2.0/sbin
> so you can run stop-yarn.sh followed by start-yarn.sh but there may not be
> any need to do so.
>
> Your values are probably being overridden by values in ~/.dt/dt-site.xml
>
> If you create config file called, say, foo.xml under
> <your-application>/src/site/conf and
> use that when you launch the application (by selecting "Use saved
> configuration"
> in the popup) its values will override those in dt-site.xml
>
> Ram
>
> On Mon, Feb 15, 2016 at 6:06 AM, Jim <jim@facility.supplies> wrote:
>
>> Tushar,
>>
>>
>>
>> I found the yarn-site.xml, but how do I restart the yarn service?
>>
>>
>>
>> Does ‘restart the gateway’ under Configuration do that?
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Jim
>>
>>
>>
>> *From:* Tushar Gosavi [mailto:tushar@datatorrent.com]
>> *Sent:* Monday, February 15, 2016 7:46 AM
>> *To:* users@apex.incubator.apache.org
>> *Subject:* Re: Application Memory Reducing
>>
>>
>>
>> Hi Jim,
>>
>>
>>
>> Can you check your yarn-site.xml for
>> yarn.scheduler.minimum-allocation-mb, it may be set to 1G (default for 2.7
>> hadoop).
>>
>> Reduce this value to 256MB and restart yarn service and try again running
>> the application.
>>
>>
>>
>>     <property>
>>
>>         <name>yarn.scheduler.minimum-allocation-mb</name>
>>
>>         <value>256</value>
>>
>>     </property>
>>
>>
>>
>> Regards,
>>
>> -Tushar.
>>
>>
>>
>>
>>
>> On Mon, Feb 15, 2016 at 7:06 PM, Jim <jim@facility.supplies> wrote:
>>
>> I have successfully built my first test Apex package on the most recent
>> Sandbox, but only one half of it can start because of memory allocation.
>>
>>
>>
>> From the article:
>>
>>
>>
>>
>> http://docs.datatorrent.com/tutorials/topnwords-c7/#managing-memory-allocation-for-containers
>>
>>
>>
>> I have put the following into my properties.xml:
>>
>>
>>
>> <*property*>
>>   <*name*>dt.application.*.attr.MASTER_MEMORY_MB</*name*>
>>   <*value*>1024</*value*>
>> </*property*>
>> <*property*>
>>   <*name*>dt.application.*.operator.*.attr.MEMORY_MB</*name*>
>>   <*value*>200</*value*>
>> </*property*>
>> <*property*>
>>   <*name*>dt.application.*.operator.*.port.*.attr.BUFFER_MEMORY_MB</
>> *name*>
>>   <*value*>200</*value*>
>> </*property*>
>>
>>
>>
>> Yet the memory allocated to my operators is still 1 GB each for the most
>> part (1 exception has 2 GB), as can be seen below:
>>
>>
>>
>>
>>
>> I would like to get this down to 512MB per operator – what is not
>> allowing the settings to take?
>>
>>
>>
>> I did see a comment where it may be the yarn configuration, but I don’t
>> know where that is within the Datatorrent Apex Sandbox configuration; nor
>> do I know if there is anything else that could be causing this?
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Jim
>>
>>
>> Jim
>>
>>
>> jim@facility.supplies (414) 760-7711
>> ------------------------------
>>
>> *The information contained in this communication, including any files or
>> attachments transmitted with it, may contain copyrighted information or
>> information that is confidential and exempt from disclosure under
>> applicable laws and regulations, is intended only for the use of the
>> recipient(s) named above, and may be legally privileged. If the reader of
>> this message is not the intended recipient, you are hereby notified that
>> any dissemination, distribution, or copying of this communication, or any
>> of its contents, files or attachments, is strictly prohibited. If you have
>> received this communication in error, please return it to the sender
>> immediately and delete the original message and any copy of it from your
>> computer system. If you have any questions concerning this message, please
>> contact the sender. *
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------
>>
>> *The information contained in this communication, including any files or
>> attachments transmitted with it, may contain copyrighted information or
>> information that is confidential and exempt from disclosure under
>> applicable laws and regulations, is intended only for the use of the
>> recipient(s) named above, and may be legally privileged. If the reader of
>> this message is not the intended recipient, you are hereby notified that
>> any dissemination, distribution, or copying of this communication, or any
>> of its contents, files or attachments, is strictly prohibited. If you have
>> received this communication in error, please return it to the sender
>> immediately and delete the original message and any copy of it from your
>> computer system. If you have any questions concerning this message, please
>> contact the sender. *
>>
>> Jim
>>
>>
>> jim@facility.supplies (414) 760-7711
>> ------------------------------
>> The information contained in this communication, including any files or
>> attachments transmitted with it, may contain copyrighted information or
>> information that is confidential and exempt from disclosure under
>> applicable laws and regulations, is intended only for the use of the
>> recipient(s) named above, and may be legally privileged. If the reader of
>> this message is not the intended recipient, you are hereby notified that
>> any dissemination, distribution, or copying of this communication, or any
>> of its contents, files or attachments, is strictly prohibited. If you have
>> received this communication in error, please return it to the sender
>> immediately and delete the original message and any copy of it from your
>> computer system. If you have any questions concerning this message, please
>> contact the sender.
>>
>
>

Mime
View raw message