flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "欧阳苗" <a952064...@qq.com>
Subject 回复: flink 内存溢出 如何排查
Date Mon, 06 Apr 2020 07:29:40 GMT
你好,
&nbsp; 我看了下这篇文档,但是只给出了加大jvm的内存,但是我更想知道的是该如何排查哪一部分代码造成的内存溢出,请问有什么方法吗。
&nbsp;其次我这里的环境是,flink on k8s, 单机版。有一个服务是专门远程flink
job,这个服务也是部署在k8s中




------------------&nbsp;原始邮件&nbsp;------------------
发件人:&nbsp;"Xintong Song"<tonysong820@gmail.com&gt;;
发送时间:&nbsp;2020年4月6日(星期一) 下午2:56
收件人:&nbsp;"user-zh"<user-zh@flink.apache.org&gt;;

主题:&nbsp;Re: flink 内存溢出 如何排查



你好,

请参考官方文档:部署与运维/内存配置/常见问题/OutOfMemoryError: Java heap
space
https://ci.apache.org/projects/flink/flink-docs-release-1.10/zh/ops/memory/mem_trouble.html#outofmemoryerror-java-heap-space

Thank you~

Xintong Song



On Mon, Apr 6, 2020 at 12:33 AM 欧阳苗 <a952064610@qq.com&gt; wrote:

&gt; 2020-04-04 18:58:41.515 ERROR 1 --- [ent-IO-thread-4]
&gt; org.apache.flink.runtime.rest.RestClient.parseResponse:393 : Received
&gt; response was neither of the expected type ([simple type, class
&gt; org.apache.flink.runtime.rest.messages.job.JobExecutionResultResponseBody])
&gt; nor an error.
&gt; Response=JsonResponse{json={"status":{"id":"COMPLETED"},"job-execution-result":{"id":"95bfafbc9b8e31565d44ae0018a0af7d","application-status":"FAILED","accumulator-results":{},"net-runtime":3098,"failure-cause":{"class":"java.lang.OutOfMemoryError","stack-trace":"java.lang.OutOfMemoryError:
&gt; Java heap
&gt; space\n","serialized-throwable":"rO0ABXNyAClvcmcuYXBhY2hlLmZsaW5rLnV0aWwuU2VyaWFsaXplZFRocm93YWJsZWUWnfUfpxPzAgADTAAZZnVsbFN0cmluZ2lmaWVkU3RhY2tUcmFjZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAFm9yaWdpbmFsRXJyb3JDbGFzc05hbWVxAH4AAVsAE3NlcmlhbGl6ZWRFeGNlcHRpb250AAJbQnhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0fPho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdlcQB+AAFbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBwdAAPSmF2YSBoZWFwIHNwYWNldXIAHltMamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50OwJGKjw8/SI5AgAAeHAAAAAAc3IAJmphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVMaXN0/A8lMbXsjhACAAFMAARsaXN0cQB+AAd4cgAsamF2YS51dGlsLkNvbGxlY3Rpb25zJFVubW9kaWZpYWJsZUNvbGxlY3Rpb24ZQgCAy173HgIAAUwAAWN0ABZMamF2YS91dGlsL0NvbGxlY3Rpb247eHBzcgATamF2YS51dGlsLkFycmF5TGlzdHiB0h2Zx2GdAwABSQAEc2l6ZXhwAAAAAHcEAAAAAHhxAH4AEXh0ACxqYXZhLmxhbmcuT3V0T2ZNZW1vcnlFcnJvcjogSmF2YSBoZWFwIHNwYWNlCnQAGmphdmEubGFuZy5PdXRPZk1lbW9yeUVycm9ydXIAAltCrPMX+AYIVOACAAB4cAAAAf6s7QAFc3IAGmphdmEubGFuZy5PdXRPZk1lbW9yeUVycm9ycjG7cIiI4xUCAAB4cgAdamF2YS5sYW5nLlZpcnR1YWxNYWNoaW5lRXJyb3I5wlZUgC8OHgIAAHhyAA9qYXZhLmxhbmcuRXJyb3JFHTZWi4IOVgIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBwdAAPSmF2YSBoZWFwIHNwYWNldXIAHltMamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50OwJGKjw8/SI5AgAAeHAAAAABc3IAG2phdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudGEJxZomNt2FAgAESQAKbGluZU51bWJlckwADmRlY2xhcmluZ0NsYXNzcQB+AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfgAFeHCAAAAAdAAAcHEAfgAOcHg="}}},
&gt; httpResponseStatus=200 OK}
&gt;
&gt;
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException:
&gt; Unrecognized field "status" (class
&gt; org.apache.flink.runtime.rest.messages.ErrorResponseBody), not marked as
&gt; ignorable (one known property: "errors"])
&gt; &amp;nbsp;at [Source: N/A; line: -1, column: -1] (through reference chain:
&gt; org.apache.flink.runtime.rest.messages.ErrorResponseBody["status"])
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:62)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:851)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1085)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1392)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperties(BeanDeserializerBase.java:1346)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:455)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1127)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:298)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:133)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:3779)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2050)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:2547)
&gt; ~[flink-shaded-jackson-2.7.9-6.0.jar!/:2.7.9-6.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.runtime.rest.RestClient.parseResponse(RestClient.java:388)
&gt; ~[flink-runtime_2.12-1.8.0.jar!/:1.8.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; org.apache.flink.runtime.rest.RestClient.lambda$submitRequest$3(RestClient.java:373)
&gt; ~[flink-runtime_2.12-1.8.0.jar!/:1.8.0]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952)
&gt; [?:1.8.0_192]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
&gt; [?:1.8.0_192]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
&gt; [?:1.8.0_192]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
&gt; [?:1.8.0_192]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
&gt; java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
&gt; [?:1.8.0_192]
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
java.lang.Thread.run(Thread.java:748) [?:1.8.0_192]
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message