apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vlad Rozov <v.ro...@datatorrent.com>
Subject Re: Debugging operator latency.
Date Tue, 21 Feb 2017 16:59:12 GMT
The name of the property is dt.attr. CONTAINER_JVM_OPTIONS. You don't 
need to replace <LOG_DIR> with the actual path as it is not known till 
an application and a containers are allocated by Yarn (Apex will replace 
<LOG_DIR> with the container log directory). You may also set the 
attribute at the launch time: 
-Ddt.attr.CONTAINER_JVM_OPTIONS="-Xloggc:<LOG_DIR>/gc.log -verbose:gc 
-XX:+PrintGCDateStamps" or using DT RTS console.

Thank you,

Vlad

/Join us at Apex Big Data World-San Jose 
<http://www.apexbigdata.com/san-jose.html>, April 4, 2017/
http://www.apexbigdata.com/san-jose-register.html 
<http://www.apexbigdata.com/san-jose-register.html>
On 2/21/17 00:44, Ambarish Pande wrote:
> I cant find the gc.log file in container logs folder. Do i have to 
> replace the LOG_DIR with a path?
>
> On Tue, Feb 21, 2017 at 12:21 PM, Tushar Gosavi 
> <tushar@datatorrent.com <mailto:tushar@datatorrent.com>> wrote:
>
>     Hi Ambarish,
>
>     you could add following property in your applications
>     properties.xml file.
>
>     <property>
>     <name>dt.application.*.attr.containerJvmOpts</name>
>     <value>-Xloggc:&lt;LOG_DIR&gt;/gc.log -verbose:gc
>     -XX:+PrintGCDateStamps</value>
>       </property>
>
>     when this property is used, you will see gc.log file in container
>     directory.
>
>     - Tushar.
>
>
>     On Tue, Feb 21, 2017 at 12:06 PM, Ambarish Pande
>     <ambarish.pande2011@gmail.com
>     <mailto:ambarish.pande2011@gmail.com>> wrote:
>     > Hello,
>     >
>     > I tried enabling gclogs in hadoop configurations. Do I need to
>     enable it in
>     > Datatorrent RTS somewhere or in my app?. If so, how should I do it?
>     >
>     > Thank You
>     >
>     > On Fri, Feb 17, 2017 at 1:20 PM, Ashwin Chandra Putta
>     > <ashwinchandrap@gmail.com <mailto:ashwinchandrap@gmail.com>> wrote:
>     >>
>     >> It is probably not able to keep up with the throughput and
>     might need more
>     >> partitions. Check the CPU and memory utilization of its
>     container. If memory
>     >> allocated to container is too low, it might be hitting GC too
>     often. You can
>     >> enable GC logging and check GC logs.
>     >>
>     >> Regards,
>     >> Ashwin.
>     >>
>     >>
>     >> On Feb 16, 2017 11:12 PM, "Ambarish Pande"
>     <ambarish.pande2011@gmail.com <mailto:ambarish.pande2011@gmail.com>>
>     >> wrote:
>     >>
>     >> Hello,
>     >>
>     >> I wanted to know why my operator latency is increasing. Which
>     logs should
>     >> I check to get any idea about that. I have checked the
>     container dt.log ,
>     >> stderr and stdout, but I cannot find anything there.
>     >>
>     >> Thank You
>     >>
>     >>
>     >
>
>


Mime
View raw message