flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ouywl <ou...@139.com>
Subject Re: Flink on Kubernetes seems to ignore log4j.properties
Date Thu, 12 Dec 2019 12:39:45 GMT
<html>
<head>
    <meta http-equiv='Content-Type' content='text/html; charset=UTF-8'>
</head>
<body>
<style>
    font{
        line-height: 1.6;
    }
    ul,ol{
        padding-left: 20px;
        list-style-position: inside;
    }
</style>
<div style = 'font-family:Helvetica,Helvetica,微软雅黑, 宋体; line-height:1.6;'>
    <div ></div><div>
    <div>&nbsp;<span contenteditable="false" ntes_mm_email="li.peng@doordash.com"
ondragstart="event.preventDefault ? event.preventDefault() : event.returnValue = false;return
false;" style="font-size:14px;color:#31353b;padding:1px 2px;">@Li Peng</span>&nbsp;<br></div><div>&nbsp;
&nbsp;I found your problems. &nbsp;Your start cmd use args “<span style="color:
rgb(106, 135, 89); font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43);">start-foreground</span>”,
It will run “<span style="font-family: Menlo; font-size: 9pt; background-color: rgb(43,
43, 43); color: rgb(197, 118, 51); font-style: italic;">exec </span><span style="font-family:
Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(106, 135, 89);">"</span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(204,
120, 50);">$</span><span style="color: rgb(169, 183, 198); font-family: Menlo;
font-size: 9pt; background-color: rgb(43, 43, 43);">{FLINK_BIN_DIR}</span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(106,
135, 89);">"</span><span style="color: rgb(169, 183, 198); font-family: Menlo;
font-size: 9pt; background-color: rgb(43, 43, 43);">/flink-console.sh </span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(204,
120, 50);">$</span><span style="color: rgb(169, 183, 198); font-family: Menlo;
font-size: 9pt; background-color: rgb(43, 43, 43);">{ENTRY_POINT_NAME} </span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(106,
135, 89);">"</span><span style="font-family: Menlo; font-size: 9pt; background-color:
rgb(43, 43, 43); color: rgb(204, 120, 50);">$</span><span style="color: rgb(169,
183, 198); font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43);">{ARGS[@]}</span><font
color="#6a8759" face="Menlo"><span style="caret-color: rgb(106, 135, 89); font-size:
12px;">”</span></font>”, and In ' flink-console.sh’, the code is “<span
style="color: rgb(169, 183, 198); font-family: Menlo; font-size: 9pt; background-color: rgb(43,
43, 43);">log_setting=(</span><span style="font-family: Menlo; font-size: 9pt;
background-color: rgb(43, 43, 43); color: rgb(106, 135, 89);">"-Dlog4j.configuration=file:</span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(204,
120, 50);">$</span><span style="color: rgb(169, 183, 198); font-family: Menlo;
font-size: 9pt; background-color: rgb(43, 43, 43);">{FLINK_CONF_DIR}</span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(106,
135, 89);">/log4j-console.properties" "-Dlogback.configurationFile=file:</span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(204,
120, 50);">$</span><span style="color: rgb(169, 183, 198); font-family: Menlo;
font-size: 9pt; background-color: rgb(43, 43, 43);">{FLINK_CONF_DIR}</span><span
style="font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43); color: rgb(106,
135, 89);">/logback-console.xml"</span><span style="color: rgb(169, 183, 198);
font-family: Menlo; font-size: 9pt; background-color: rgb(43, 43, 43);">)</span>”
. So the log4j.properties not work. It need log4j-console.properties and logback-console.xml.</div>
    <div>
        <span>
            <br>
        </span>
    </div>
    <div id="ntes-pcmac-signature" style="font-family:'Helvetica','Microsoft Yahei', '微软雅黑'">
      
    <div style="font-size:14px; padding: 0;  margin:0;line-height: 14px;">
        <div style="border-bottom:1px solid #e6e6e6;display:inline-block;">
                    <a href="https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&amp;name=ouywl&amp;uid=ouywl%40139.com&amp;iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsma8dc7719018ba2517da7111b3db5a170.jpg&amp;items=%5B%22ouywl%40139.com%22%5D"
style="display:block;background:#fff; max-width: 400px; _width: 400px;padding:15px 0 10px
0;text-decoration: none; outline:none;-webkit-tap-highlight-color:transparent;-webkit-text-size-adjust:none
!important;text-size-adjust:none !important;">
            <table cellpadding="0" style="width: 100%; max-width: 100%; table-layout: fixed;
border-collapse: collapse;color: #9b9ea1;font-size: 14px;line-height:1.3;-webkit-text-size-adjust:none
!important;text-size-adjust:none !important;">
                <tbody style="font-family: 'PingFang SC', 'Hiragino Sans GB','WenQuanYi
Micro Hei', 'Microsoft Yahei', '微软雅黑', verdana !important; word-wrap:break-word; word-break:break-all;-webkit-text-size-adjust:none
!important;text-size-adjust:none !important;">
                    <tr class="firstRow">
                            <td width="38" style="padding:0; box-sizing: border-box; width:
38px;">
                                <img width="38" height="38" style="vertical-align:middle;
width: 38px; height: 38px; border-radius:50%;" src="https://mail-online.nosdn.127.net/sma8dc7719018ba2517da7111b3db5a170.jpg">
                            </td>
                            <td style="padding: 0 0 0 10px; color: #31353b;">
                                <div style="font-size: 16px;font-weight:bold; width:100%;
white-space: nowrap; overflow:hidden;text-overflow: ellipsis;">ouywl</div>
                            </td>
                    </tr>
                        <tr width="100%" style="font-size: 14px !important; width: 100%;">
                            <td colspan="2" style="padding:10px 0 0 0; font-size:14px !important;
width: 100%;">
                                    <div style="width: 100%;font-size: 14px !important;word-wrap:break-word;word-break:break-all;">ouywl@139.com</div>
                            </td>
                        </tr>
                </tbody>
            </table>
        </a>
        </div>
    </div>
    <div style="font-size:12px;color:#b5b9bd;margin-top:8px;line-height: 18px;">
        <span>签名由</span>
        <a style="text-decoration: none;color:#4196ff;padding:0px;" href="https://mail.163.com/dashi/dlpro.html?from=mail81">网易邮箱大师</a>
        <span>定制</span>
    </div>
 </div>
    <br>
</div><div class="J-reply" style="background-color:#f2f2f2;color:black;padding-top:6px;padding-bottom:6px;border-radius:3px;-moz-border-radius:3px;-webkit-border-radius:3px;margin-top:45px;margin-bottom:20px;font-family:''Helvetica','Microsoft
Yahei', '微软雅黑'';">
    <div style="font-size:12px;line-height:1.6;word-break:break-all;margin-left:10px;margin-right:10px">On
<span class="mail-date">12/12/2019 15:35</span>,<a class="mail-to" style="text-decoration:none;color:#2a83f2;"
href="mailto:ouywl@139.com">ouywl&lt;ouywl@139.com&gt;</a> wrote: </div>
</div>
<blockquote id="ntes-pcmail-quote" style="margin: 0; padding: 0; font-size: 14px; font-family:
''Helvetica','Microsoft Yahei', '微软雅黑'';">

<style>
    font{
        line-height: 1.6;
    }
    ul,ol{
        padding-left: 20px;
        list-style-position: inside;
    }
</style>
<div style="font-family:Helvetica,Helvetica,微软雅黑, 宋体; line-height:1.6;">
    <div></div><div>
    <div>HI yang,</div><div>&nbsp; &nbsp;Could you give more info
detail? log4j.properties content, and The k8s yaml. Is use the dockerfile in flink-container?
When I test it use the default per-job yaml in flick-container? It is only show logs in docker
infos. And not logs in /opt/flink/log.</div>
    <div>
        <span>
            <br>
        </span>
    </div>
    <div id="ntes-pcmac-signature" style="font-family:'Helvetica','Microsoft Yahei', '微软雅黑'">
      
    <div style="font-size:14px; padding: 0;  margin:0;line-height: 14px;">
        <div style="border-bottom:1px solid #e6e6e6;display:inline-block;">
                    <a href="https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&amp;name=ouywl&amp;uid=ouywl%40139.com&amp;iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fsma8dc7719018ba2517da7111b3db5a170.jpg&amp;items=%5B%22ouywl%40139.com%22%5D"
style="display:block;background:#fff; max-width: 400px; _width: 400px;padding:15px 0 10px
0;text-decoration: none; outline:none;-webkit-tap-highlight-color:transparent;-webkit-text-size-adjust:none
!important;text-size-adjust:none !important;">
            <table cellpadding="0" style="width: 100%; max-width: 100%; table-layout: fixed;
border-collapse: collapse;color: #9b9ea1;font-size: 14px;line-height:1.3;-webkit-text-size-adjust:none
!important;text-size-adjust:none !important;">
                <tbody style="font-family: 'PingFang SC', 'Hiragino Sans GB','WenQuanYi
Micro Hei', 'Microsoft Yahei', '微软雅黑', verdana !important; word-wrap:break-word; word-break:break-all;-webkit-text-size-adjust:none
!important;text-size-adjust:none !important;">
                    <tr class="firstRow">
                            <td width="38" style="padding:0; box-sizing: border-box; width:
38px;">
                                <img width="38" height="38" style="vertical-align:middle;
width: 38px; height: 38px; border-radius:50%;" src="https://mail-online.nosdn.127.net/sma8dc7719018ba2517da7111b3db5a170.jpg">
                            </td>
                            <td style="padding: 0 0 0 10px; color: #31353b;">
                                <div style="font-size: 16px;font-weight:bold; width:100%;
white-space: nowrap; overflow:hidden;text-overflow: ellipsis;">ouywl</div>
                            </td>
                    </tr>
                        <tr width="100%" style="font-size: 14px !important; width: 100%;">
                            <td colspan="2" style="padding:10px 0 0 0; font-size:14px !important;
width: 100%;">
                                    <div style="width: 100%;font-size: 14px !important;word-wrap:break-word;word-break:break-all;">ouywl@139.com</div>
                            </td>
                        </tr>
                </tbody>
            </table>
        </a>
        </div>
    </div>
    <div style="font-size:12px;color:#b5b9bd;margin-top:8px;line-height: 18px;">
        <span>签名由</span>
        <a style="text-decoration: none;color:#4196ff;padding:0px;" href="https://mail.163.com/dashi/dlpro.html?from=mail81">网易邮箱大师</a>
        <span>定制</span>
    </div>
 </div>
    <br>
</div><div class="J-reply" style="background-color:#f2f2f2;color:black;padding-top:6px;padding-bottom:6px;border-radius:3px;-moz-border-radius:3px;-webkit-border-radius:3px;margin-top:45px;margin-bottom:20px;font-family:''Helvetica','Microsoft
Yahei', '微软雅黑'';">
    <div style="font-size:12px;line-height:1.6;word-break:break-all;margin-left:10px;margin-right:10px">On
<span class="mail-date">12/12/2019 13:47</span>,<a class="mail-to" style="text-decoration:none;color:#2a83f2;"
href="mailto:danrtsey.wy@gmail.com">Yang Wang&lt;danrtsey.wy@gmail.com&gt;</a>
wrote: </div>
</div>
<blockquote id="ntes-pcmail-quote" style="margin: 0; padding: 0; font-size: 14px; font-family:
''Helvetica','Microsoft Yahei', '微软雅黑'';">
<div dir="ltr">Hi Peng,<div><br></div><div>What i mean is to
use `docker exec` into the running pod and `ps` to get the real</div><div>command
that is running for jobmanager.&nbsp;</div><div>Do you have checked the /opt/flink/conf/log4j.properties
is right?</div><div><br></div><div>I have tested standalone
per-job on my kubernetes cluster, the logs show up as expected.</div><div><br></div><div><br></div><div>Best,</div><div>Yang</div></div><br><div
class="gmail_quote"><div dir="ltr" class="gmail_attr">Li Peng &lt;<a href="mailto:li.peng@doordash.com">li.peng@doordash.com</a>&gt;
于2019年12月12日周四 上午2:59写道:<br></div><blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div
dir="ltr"><div>Hey Yang, here are the commands:<br><pre style="background-color:rgb(43,43,43);color:rgb(169,183,198);font-family:Menlo;font-size:9pt"><span
style="color:rgb(106,135,89)">"/opt/flink/bin/taskmanager.sh"</span>,<br><span
style="color:rgb(106,135,89)">"start-foreground"</span>,<br><span style="color:rgb(106,135,89)">"-Djobmanager.rpc.address={{
.Chart.Name }}-job-manager"</span>,<br><span style="color:rgb(106,135,89)">"-Dtaskmanager.numberOfTaskSlots=1"</span></pre></div><div><pre
style="background-color:rgb(43,43,43);color:rgb(169,183,198);font-family:Menlo;font-size:9pt"><span
style="color:rgb(106,135,89)">"/opt/flink/bin/standalone-job.sh"</span>,<br><span
style="color:rgb(106,135,89)">"start-foreground"</span>,<br><span style="color:rgb(106,135,89)">"-Djobmanager.rpc.address={{
.Chart.Name }}-job-manager"</span>,<br><span style="color:rgb(106,135,89)">"-Dparallelism.default={{
.Values.task.replicaCount }}"</span></pre></div><div>Yes it's very
curious that I don't see any logs actually written&nbsp;to /opt/flink/log.&nbsp;</div><div><br></div><div
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Dec 10, 2019 at 11:17
PM Yang Wang &lt;<a href="mailto:danrtsey.wy@gmail.com" target="_blank">danrtsey.wy@gmail.com</a>&gt;
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px
solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Could you find
the logs under /opt/flink/log/jobmanager.log? If not, please share the</div><div>commands
the JobManager and TaskManager are using? If the command is correct</div><div>and
the log4j under /opt/flink/conf is expected, it is so curious why we could not get the logs.</div><div><br></div><div><br></div><div>Best,</div><div>Yang</div></div><br><div
class="gmail_quote"><div dir="ltr" class="gmail_attr">Li Peng &lt;<a href="mailto:li.peng@doordash.com"
target="_blank">li.peng@doordash.com</a>&gt; 于2019年12月11日周三 下午1:24写道:<br></div><blockquote
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div
dir="ltr">Ah I see. I think the Flink app is reading files from /opt/flink/conf&nbsp;correctly
as it is, since changes I make to flink-conf are picked up as expected, it's just the log4j
properties that are either not being used, or don't apply&nbsp;to stdout or whatever source
k8 uses for its logs? Given that the pods don't seem to have logs written to file anywhere,&nbsp;contrary
to the properties, I'm inclined to say it's the former and that the log4j properties just&nbsp;aren't
being picked up. Still have no idea why though.</div><br><div class="gmail_quote"><div
dir="ltr" class="gmail_attr">On Tue, Dec 10, 2019 at 6:56 PM Yun Tang &lt;<a href="mailto:myasuka@live.com"
target="_blank">myasuka@live.com</a>&gt; wrote:<br></div><blockquote
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">





<div lang="EN-US">
<div>
<p class="MsoNormal">Sure, /opt/flink/conf is mounted as a volume from the configmap.<u></u><u></u></p>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<p class="MsoNormal">Best<u></u><u></u></p>
<p class="MsoNormal">Yun Tang<u></u><u></u></p>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid
rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12pt;color:black">From: </span></b><span
style="font-size:12pt;color:black">Li Peng &lt;<a href="mailto:li.peng@doordash.com"
target="_blank">li.peng@doordash.com</a>&gt;<br>
<b>Date: </b>Wednesday, December 11, 2019 at 9:37 AM<br>
<b>To: </b>Yang Wang &lt;<a href="mailto:danrtsey.wy@gmail.com" target="_blank">danrtsey.wy@gmail.com</a>&gt;<br>
<b>Cc: </b>vino yang &lt;<a href="mailto:yanghua1127@gmail.com" target="_blank">yanghua1127@gmail.com</a>&gt;,
user &lt;<a href="mailto:user@flink.apache.org" target="_blank">user@flink.apache.org</a>&gt;<br>
<b>Subject: </b>Re: Flink on Kubernetes seems to ignore log4j.properties<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">1. Hey Yun, I'm calling /opt/flink/bin/standalone-job.sh and /opt/flink/bin/taskmanager.sh
on my job and task managers respectively. It's based on the setup described here:&nbsp;<a
href="http://shzhangji.com/blog/2019/08/24/deploy-flink-job-cluster-on-kubernetes/" target="_blank">http://shzhangji.com/blog/2019/08/24/deploy-flink-job-cluster-on-kubernetes/</a>&nbsp;.
 I haven't tried the configmap approach yet, does it also replace the conf files in /opt/flink/conf?<br>
<br>
2. Hey Vino, here's a sample of the kubernetes:&nbsp;<a href="https://pastebin.com/fqJrgjZu"
target="_blank">https://pastebin.com/fqJrgjZu</a>&nbsp; I didn't change any patterns
from the default, so the string patterns should look the same, but as you can see it's full
of info checkpoint
 logs that I originally was trying to suppress. Based on my log4j.properties, the level should
be set to WARN. I couldn't actually find any .out files on the pod, this is from the kubectl
logs command. I also didn't see any files in /opt/flink/log, which I
 thought my log4j was specified to do, hence me thinking that the properties weren't actually
being consumed. I also have the same properties in my src/main/resources folder.<br>
<br>
3. Hey Yang, yes this is a standalone session cluster. I did specify in the docker file to
copy the log4j.properties to the /opt/flink/conf folder on the image, and I confirmed that
the properties are correct when I bash'd into the pod and viewed them manually.
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Incidentally, I also tried passing the&nbsp;-Dlog4j.configuration
argument to the programs, and it doesn't work either. And based on what I'm reading on jira,
that option is not really supported anymore?<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks for your responses, folks!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Li<u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Dec 9, 2019 at 7:10 PM Yang Wang &lt;<a href="mailto:danrtsey.wy@gmail.com"
target="_blank">danrtsey.wy@gmail.com</a>&gt; wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt
solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">Hi Li Peng, <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">You are running standalone session cluster or per-job cluster on
kubernetes. Right?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">If so, i think you need to check your log4j.properties in the image,
not local. The log is<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">stored to /opt/flink/log/jobmanager.log by default.&nbsp;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">If you are running active Kubernetes integration for a fresh taste.
The following cli option<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">could be used to remove the redirect.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">-Dkubernetes.container-start-command-template="%java% %classpath%
%jvmmem% %jvmopts% %logging% %class% %args%"<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Best,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Yang<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">vino yang &lt;<a href="mailto:yanghua1127@gmail.com" target="_blank">yanghua1127@gmail.com</a>&gt;
<span lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">于</span>2019<span
lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">年</span>12<span
lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">月</span>10<span
lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">日周二</span><span
lang="ZH-CN">
</span><span lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">上午</span>10:55<span
lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">写道:</span><u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt
solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">Hi Li, <u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">A potential reason could be conflicting logging frameworks.&nbsp;Can
you share the log in your .out file and let us know if the print format of the log is the
same as the configuration file you gave.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Best,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Vino<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
<div>
<div>
<p class="MsoNormal">Li Peng &lt;<a href="mailto:li.peng@doordash.com" target="_blank">li.peng@doordash.com</a>&gt;
<span lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">于</span>2019<span
lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">年</span>12<span
lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">月</span>10<span
lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">日周二</span><span
lang="ZH-CN">
</span><span lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">上午</span>10:09<span
lang="ZH-CN" style="font-family:&quot;Microsoft YaHei&quot;,sans-serif">写道:</span><u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt
solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">Hey folks, I noticed that my kubernetes flink logs (reached via&nbsp;<i>kubectl
logs &lt;pod-name&gt;</i>) completely ignore any of the configurations I put
into /flink/conf/. I set the logger level to WARN, yet I still see INFO level logging from
 flink loggers like&nbsp;org.apache.flink.runtime.checkpoint.CheckpointCoordinator. I
even made copied the same properties to&nbsp;/flink/conf/log4j-console.properties and&nbsp;log4j-cli.properties.&nbsp;
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">From what I can tell, kubernetes just listens to stdout and stderr,
so shouldn't the log4j.properties control output to them? Anyone seen this issue before?<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u>&nbsp;<u></u></p>
</div>
<div>
<p class="MsoNormal">Here is my log4j.properties:<u></u><u></u></p>
</div>
<div>
<pre style="background:rgb(43,43,43)"><span style="font-size:9pt;font-family:Menlo;color:rgb(169,183,198)"><br></span><span
style="font-size:9pt;font-family:Menlo;color:gray"># This affects logging for both user
code and Flink<br></span><span style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.rootLogger</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">WARN,
file, console, stdout<br><br></span><span style="font-size:9pt;font-family:Menlo;color:gray">#
Uncomment this if you want to _only_ change Flink's logging<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.logger.org.apache.flink</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">WARN<br><br></span><span
style="font-size:9pt;font-family:Menlo;color:gray"># The following lines keep the log level
of common libraries/connectors on<br># log level INFO. The root logger does not override
this. You have to manually<br># change the log levels here.<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.logger.akka</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">INFO<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.logger.org.apache.kafka</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">INFO<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.logger.org.apache.hadoop</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">INFO<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.logger.org.apache.zookeeper</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">INFO<br><br></span><span
style="font-size:9pt;font-family:Menlo;color:gray"># Log all infos in the given file<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.appender.file</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">org.apache.log4j.FileAppender<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.appender.file.file</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">${log.file}<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.appender.file.append</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">false<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.appender.file.layout</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">org.apache.log4j.PatternLayout<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.appender.file.layout.ConversionPattern</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">%d{yyyy-MM-dd
HH:mm:ss,SSS} %-5p %-60c %x - %m%n<br><br></span><span style="font-size:9pt;font-family:Menlo;color:gray">#
Log all infos to the console<br></span><span style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.appender.console</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">org.apache.log4j.ConsoleAppender<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.appender.console.layout</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">org.apache.log4j.PatternLayout<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.appender.console.layout.ConversionPattern</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">%d{yyyy-MM-dd
HH:mm:ss,SSS} %-5p %-60c %x - %m%n<br><br></span><span style="font-size:9pt;font-family:Menlo;color:gray">#
Suppress the irrelevant (wrong) warnings from the Netty channel handler<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.logger.org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">ERROR,
file, console<br></span><span style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.logger.org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">WARN<br></span><span
style="font-size:9pt;font-family:Menlo;color:rgb(204,120,50)">log4j.logger.org.apache.flink.runtime.checkpoint</span><span
style="font-size:9pt;font-family:Menlo;color:gray">=</span><span style="font-size:9pt;font-family:Menlo;color:rgb(106,135,89)">WARN</span><span
style="font-size:9pt;font-family:Menlo;color:rgb(169,183,198)"><u></u><u></u></span></pre>
<p class="MsoNormal">Thanks,<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal">Li<u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>

</blockquote></div>
</blockquote></div>
</blockquote></div></div>
</blockquote></div>

</blockquote><!--�-->
</div></blockquote><!--�-->
</div>
</body>
</html>
Mime
View raw message