hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Janos Matyas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1842) InvalidApplicationMasterRequestException raised during AM-requested shutdown
Date Wed, 23 Apr 2014 16:41:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978415#comment-13978415
] 

Janos Matyas commented on YARN-1842:
------------------------------------


We had the same issues on OSX (during dev) and Debian server(s), but since we switched to
CentOS it works fine (Hadoop 2.3, Hoya 0.13, HBase 0.98, Zookeeper 3.3.6).

I was hoping to find that this is some env/os related issue, and I have built a new docker
image(s) starting from the same Dockerfile we use with CentOS but I wasn't able to reproduce
it on Ubuntu. You can get the Ubuntu based Hoya image from https://github.com/matyix/hoya-docker-ubuntu
or the CentOS one from https://github.com/sequenceiq/hoya-docker if you'd like to try it.

For us the problem was coming when we were freezing HBase or Flume clusters (custom provider)
with Hoya ( hoya freeze hbase --manager localhost:8032 --filesystem hdfs://localhost:9000).

I will try it on Debian tomorrow, but I think I will need to skim through the code and see
what we have changed to get rid of this issue.

We are migrating this to Slider but as Steve mentioned the code in AM is still there ...




> InvalidApplicationMasterRequestException raised during AM-requested shutdown
> ----------------------------------------------------------------------------
>
>                 Key: YARN-1842
>                 URL: https://issues.apache.org/jira/browse/YARN-1842
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.3.0
>            Reporter: Steve Loughran
>            Priority: Minor
>         Attachments: hoyalogs.tar.gz
>
>
> Report of the RM raising a stack trace [https://gist.github.com/matyix/9596735] during
AM-initiated shutdown. The AM could just swallow this and exit, but it could be a sign of
a race condition YARN-side, or maybe just in the RM client code/AM dual signalling the shutdown.

> I haven't replicated this myself; maybe the stack will help track down the problem. Otherwise:
what is the policy YARN apps should adopt for AM's handling errors on shutdown? go straight
to an exit(-1)?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message