hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15570) LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
Date Fri, 27 Jan 2017 08:58:25 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15842411#comment-15842411
] 

Siddharth Seth commented on HIVE-15570:
---------------------------------------

I don't think we should fallback to using locations provided by the FS, if daemons are not
available. That will pollute the cache once the daemons come up - for subsequent queries.
Think it's better to fail in this case, with a message saying that LLAP is not running.
There's no check in HS2 at the moment to see if LLAP is up before handing it off to an AM
for execution. Building that out would be useful. Once that is in place, the SplitLocationProvider
could wait for LLAP to come up (with a timeout) before proceeding.

[~sershe] - thoughts?

> LLAP: Exception in HostAffinitySplitLocationProvider when running in container mode
> -----------------------------------------------------------------------------------
>
>                 Key: HIVE-15570
>                 URL: https://issues.apache.org/jira/browse/HIVE-15570
>             Project: Hive
>          Issue Type: Bug
>          Components: llap
>            Reporter: Rajesh Balamohan
>            Assignee: Zhiyuan Yang
>            Priority: Minor
>         Attachments: HIVE-15570.1.patch, HIVE-15570.2.patch
>
>
> Sometimes user might prefer to run with "hive.execution.mode=container" mode when LLAP
is stopped. If hive config for LLAP had "hive.llap.client.consistent.splits=true" in client
side, it would end up throwing the following exception in {{Utils.java}}.
> {noformat}
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>         at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
>         ... 25 more
> Caused by: java.lang.IllegalStateException: org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider
needs at least 1 location to function
>         at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
>         at org.apache.hadoop.hive.ql.exec.tez.HostAffinitySplitLocationProvider.<init>(HostAffinitySplitLocationProvider.java:52)
>         at org.apache.hadoop.hive.ql.exec.tez.Utils.getSplitLocationProvider(Utils.java:54)
>         at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:121)
>         ... 30 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message