flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hilmi Yildirim <Hilmi.Yildi...@dfki.de>
Subject Re: Flink Yarn Stack Size
Date Fri, 22 Jan 2016 13:18:12 GMT
Hi Robert,
in the logs I found this

13:54:41,291 INFO 
org.apache.flink.yarn.YarnTaskManagerRunner                   -  JVM 
Options:
13:54:41,292 INFO 
org.apache.flink.yarn.YarnTaskManagerRunner                   - -Xms3072m
13:54:41,292 INFO 
org.apache.flink.yarn.YarnTaskManagerRunner                   - -Xmx3072m
13:54:41,292 INFO 
org.apache.flink.yarn.YarnTaskManagerRunner                   - 
-XX:MaxDirectMemorySize=3072m
13:54:41,292 INFO 
org.apache.flink.yarn.YarnTaskManagerRunner                   - -Xss1024m
13:54:41,292 INFO 
org.apache.flink.yarn.YarnTaskManagerRunner                   - 
-Dlog.file=/var/log/hadoop-yarn/container/application_1453384265041_0030/container_1453384265041_0030_01_000002/taskmanager.log
13:54:41,292 INFO 
org.apache.flink.yarn.YarnTaskManagerRunner                   - 
-Dlogback.configurationFile=file:logback.xml
13:54:41,292 INFO 
org.apache.flink.yarn.YarnTaskManagerRunner                   - 
-Dlog4j.configuration=file:log4j.properties


It seems that the argument is passed to the Taskmanager. Or am I wrong?

Best Regards,
Hilmi


Am 22.01.2016 um 12:25 schrieb Robert Metzger:
> Hi Hilmi,
>
> it means that each JVM the YARN session is starting (JobManager and
> TaskManagers) is initialized with that parameter.
>
> Can you check the logs of the TaskManager to see if the option has been
> passed properly?
>
> On Fri, Jan 22, 2016 at 12:22 PM, Hilmi Yildirim <Hilmi.Yildirim@dfki.de>
> wrote:
>
>> Hi,
>> I added the following line to my flink-conf.yaml:
>>
>> env.java.opts: "-Xss1024m"
>>
>> But it does not work. Does this line mean that every single slot of each
>> task manager uses a stack size of 1 GB?
>>
>> Best Regards,
>> Hilmi
>>
>> Am 22.01.2016 um 11:46 schrieb Robert Metzger:
>>
>>> Hi Hilmi,
>>>
>>> Flink on YARN is respecting the env.java.opts configuration parameter from
>>> the flink-conf.yaml.
>>>
>>> 2016-01-22 11:44 GMT+01:00 Hilmi Yildirim <Hilmi.Yildirim@dfki.de>:
>>>
>>> Hi,
>>>> does anybody know how I can increase the stack size of a yarn session?
>>>>
>>>> Best Regards,
>>>> Hilmi
>>>>
>>>> --
>>>> ==================================================================
>>>> Hilmi Yildirim, M.Sc.
>>>> Researcher
>>>>
>>>> DFKI GmbH
>>>> Intelligente Analytik für Massendaten
>>>> DFKI Projektbüro Berlin
>>>> Alt-Moabit 91c
>>>> D-10559 Berlin
>>>> Phone: +49 30 23895 1814
>>>>
>>>> E-Mail: Hilmi.Yildirim@dfki.de
>>>>
>>>> -------------------------------------------------------------
>>>> Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
>>>> Firmensitz: Trippstadter Strasse 122, D-67663 Kaiserslautern
>>>>
>>>> Geschaeftsfuehrung:
>>>> Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>>>> Dr. Walter Olthoff
>>>>
>>>> Vorsitzender des Aufsichtsrats:
>>>> Prof. Dr. h.c. Hans A. Aukes
>>>>
>>>> Amtsgericht Kaiserslautern, HRB 2313
>>>> -------------------------------------------------------------
>>>>
>>>>
>>>>
>> --
>> ==================================================================
>> Hilmi Yildirim, M.Sc.
>> Researcher
>>
>> DFKI GmbH
>> Intelligente Analytik für Massendaten
>> DFKI Projektbüro Berlin
>> Alt-Moabit 91c
>> D-10559 Berlin
>> Phone: +49 30 23895 1814
>>
>> E-Mail: Hilmi.Yildirim@dfki.de
>>
>> -------------------------------------------------------------
>> Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
>> Firmensitz: Trippstadter Strasse 122, D-67663 Kaiserslautern
>>
>> Geschaeftsfuehrung:
>> Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>> Dr. Walter Olthoff
>>
>> Vorsitzender des Aufsichtsrats:
>> Prof. Dr. h.c. Hans A. Aukes
>>
>> Amtsgericht Kaiserslautern, HRB 2313
>> -------------------------------------------------------------
>>
>>


-- 
==================================================================
Hilmi Yildirim, M.Sc.
Researcher

DFKI GmbH
Intelligente Analytik für Massendaten
DFKI Projektbüro Berlin
Alt-Moabit 91c
D-10559 Berlin
Phone: +49 30 23895 1814

E-Mail: Hilmi.Yildirim@dfki.de

-------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Strasse 122, D-67663 Kaiserslautern

Geschaeftsfuehrung:
Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
Dr. Walter Olthoff

Vorsitzender des Aufsichtsrats:
Prof. Dr. h.c. Hans A. Aukes

Amtsgericht Kaiserslautern, HRB 2313
-------------------------------------------------------------


Mime
View raw message