hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-18269) LLAP: Fast llap io with slow processing pipeline can lead to OOM
Date Wed, 13 Dec 2017 09:21:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-18269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Prasanth Jayachandran updated HIVE-18269:
-----------------------------------------
    Attachment: Screen Shot 2017-12-13 at 1.15.16 AM.png

Attached images show the retained references. Entire 40GB heap was occupied by pendingData
(some nodes are not expanded for brevity)

> LLAP: Fast llap io with slow processing pipeline can lead to OOM
> ----------------------------------------------------------------
>
>                 Key: HIVE-18269
>                 URL: https://issues.apache.org/jira/browse/HIVE-18269
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: Screen Shot 2017-12-13 at 1.15.16 AM.png
>
>
> pendingData linked list in Llap IO elevator (LlapRecordReader.java) may have grow indefinitely
when Llap IO is faster than processing pipeline. Since we don't have backpressure to slow
down the IO, this can lead to indefinite growth of pending data leading to severe GC pressure
and eventually lead to OOM.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message