hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gunther Hagleitner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10162) LLAP: Avoid deserializing the plan > 1 times in a single thread
Date Thu, 02 Apr 2015 21:19:54 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393446#comment-14393446
] 

Gunther Hagleitner commented on HIVE-10162:
-------------------------------------------

yeah, i broke that. i thought that we won't be able to use thread locals for plan objects
anymore. turns out we still use one thread per query fragment. MRLegacyInput makes sure that
the same thread is used for both input and processor and apparently we currently start a new
thread per child.

long story short - i can switch back to the thread local model for now. however, that's brittle,
stupid and inefficient. usually we only allow two of those :-)

> LLAP: Avoid deserializing the plan > 1 times in a single thread
> ---------------------------------------------------------------
>
>                 Key: HIVE-10162
>                 URL: https://issues.apache.org/jira/browse/HIVE-10162
>             Project: Hive
>          Issue Type: Sub-task
>    Affects Versions: llap
>            Reporter: Gopal V
>            Assignee: Gunther Hagleitner
>             Fix For: llap
>
>         Attachments: deserialize-plan-1.png, deserialize-plan-2.png
>
>
> Kryo shows up in the critical hot-path for LLAP when using a plan with a very large filter
condition, due to the fact that the plan is deserialized more than once for each task.
> !deserialize-plan-1.png!
> !deserialize-plan-2.png!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message