hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gagan Brahmi <gaganbra...@gmail.com>
Subject Re: Yarn REST API - cannot kill an application
Date Wed, 06 Apr 2016 04:53:25 GMT
You are using the wrong method to "KILL" the application. POST is used
to submit the application to RM.

You need to use PUT method to kill an application.

Here's how I would use the REST API to kill the application:

curl -H "Content-Type: application/json" -X PUT -d
'{"state":"KILLED"}'
'http://<RM_HOST>:8088/ws/v1/cluster/apps/<APP_ID>/state'


Regards,
Gagan Brahmi

On Tue, Apr 5, 2016 at 2:52 AM, Tallanel Siriel
<tallanel.hadoop@gmail.com> wrote:
> Hello !
>
> I send you this mail because I have a problem when I try to kill an
> application I just submit.
> I have Hadoop 2.7.1 (HDP 2.3.2).
>
> Pre-requisite :
> - My Resource Manager is in HA
> - My cluster is Kerberized
>
> Execution of a job (I wordcount a logfile, it does not really matter) :
> ###
> hadoop jar
> /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar
> wordcount -Dmapreduce.job.queuename=<a yarn queue> /tmp/ambari-server.log.9
> /tmp/myresult/
> ###
>
> While the job is running, I try to kill it :
> ###
> curl --negotiate -ikv -u: -X POST -d '{"state": "KILLED"}' 'http://<ACTIVE
> RM>:8088/ws/v1/cluster/apps/<APPLICATION_ID>'
> ###
>
> And here is the log :
> ###
> * About to connect() to <ACTIVE RM> port 8088 (#0)
> * Trying <ACTIVE_RM_IP>... connected
> * Connected to <ACTIVE RM> (<ACTIVE_RM_IP>) port 8088 (#0)
>> POST /ws/v1/cluster/apps/<APPLICATION_ID> HTTP/1.1
>> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7
>> NSS/3.16.2.3 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
>> Host: <ACTIVE RM>:8088
>> Accept: */*
>> Content-Length: 19
>> Content-Type: application/x-www-form-urlencoded
>>
> < HTTP/1.1 500 Internal Server Error
> HTTP/1.1 500 Internal Server Error
> < Cache-Control: no-cache
> Cache-Control: no-cache
> < Expires: Tue, 05 Apr 2016 08:56:30 GMT
> Expires: Tue, 05 Apr 2016 08:56:30 GMT
> < Date: Tue, 05 Apr 2016 08:56:30 GMT
> Date: Tue, 05 Apr 2016 08:56:30 GMT
> < Pragma: no-cache
> Pragma: no-cache
> < Expires: Tue, 05 Apr 2016 08:56:30 GMT
> Expires: Tue, 05 Apr 2016 08:56:30 GMT
> < Date: Tue, 05 Apr 2016 08:56:30 GMT
> Date: Tue, 05 Apr 2016 08:56:30 GMT
> < Pragma: no-cache
> Pragma: no-cache
> < Content-Type: application/xml
> Content-Type: application/xml
> < Content-Length: 202
> Content-Length: 202
> < Server: Jetty(6.1.26.hwx)
> Server: Jetty(6.1.26.hwx)
>
> <
> * Connection #0 to host <ACTIVE RM> left intact
> * Closing connection #0
> <?xml version="1.0" encoding="UTF-8"
> standalone="yes"?><RemoteException><exception>WebApplicationException</exception><javaClassName>javax.ws.rs.WebApplicationException</javaClassName></RemoteException>
> ###
>
> And in the logs of the RM, I can see the following :
> ###
> 2016-04-05 11:05:05,193 INFO  rmcontainer.RMContainerImpl
> (RMContainerImpl.java:handle(417)) -
> container_e36_1459845370345_0006_01_000002 Container Transitioned from
> ACQUIRED to RUNNING
> 2016-04-05 11:05:05,518 INFO  scheduler.AppSchedulingInfo
> (AppSchedulingInfo.java:updateResourceRequests(185)) - checking for
> deactivate of application : <APPLICATION_ID>
> 2016-04-05 11:05:07,733 WARN  webapp.GenericExceptionHandler
> (GenericExceptionHandler.java:toResponse(98)) - INTERNAL_SERVER_ERROR
> javax.ws.rs.WebApplicationException
>         at
> com.sun.jersey.server.impl.uri.rules.TerminatingRule.accept(TerminatingRule.java:66)
>         at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
>         at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>         at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
>         at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
>         at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
>         at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
>         at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
>         at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
>         at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
>         at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:886)
>         at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
>         at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:142)
>         at
> com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
>         at
> com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
>         at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
>         at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
>         at
> com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:595)
>         at
> org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:554)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1243)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>         at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>         at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>         at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
>         at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>         at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>         at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>         at org.mortbay.jetty.Server.handle(Server.java:326)
>         at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>         at
> org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
>         at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
>         at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
>         at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>         at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
>         at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> 2016-04-05 11:05:16,274 INFO  rmcontainer.RMContainerImpl
> (RMContainerImpl.java:handle(417)) -
> container_e36_1459845370345_0006_01_000002 Container Transitioned from
> RUNNING to COMPLETED
> ###
>
> Does anyone has this problem with HA RM ?
> I noticed that I can normally kill the application using Yarn CLI, but I
> cannot using the Yarn REST API.
>
> BR.
>
> Tallanel.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org


Mime
View raw message