livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Bozarth" <ajboz...@us.ibm.com>
Subject Re: How to cancel the running streaming job using livy?
Date Sat, 20 Jan 2018 00:03:07 GMT

Ah, that's why I couldn't find cancel in JobHandle, but it was implemented
in all it's implementations, which all implement it as would be expected.

                                                                                   
 Alex Bozarth                                                                      
 Software Engineer                                                                 
 Spark Technology Center                                                           
                                                                                   
                                                                                     
                                                                                     
                                                                                     
 E-mail: ajbozart@us.ibm.com                                                         
 GitHub: github.com/ajbozarth                                                        
                                                                   505 Howard Street 
                                                             San Francisco, CA 94105 
                                                                       United States 
                                                                                     








From:	Marcelo Vanzin <vanzin@cloudera.com>
To:	user@livy.incubator.apache.org
Date:	01/19/2018 03:55 PM
Subject:	Re: How to cancel the running streaming job using livy?



A JobHandle (which you get by submitting a Job) is a Future, and Futures
have a "cancel()" method.

I don't remember the details about how "cancel()" is implemented in Livy,
though.

On Fri, Jan 19, 2018 at 3:52 PM, Alex Bozarth <ajbozart@us.ibm.com> wrote:
  Ok so I looked into this a bit more. I misunderstood you a bit before,
  the delete call is for ending livy sessions using the rest API, not jobs
  and not via the Java API. As for the Job state that makes sense, if you
  end the session the session kills all currently running jobs. What you
  want to to send cancel requests to the jobs the session is running. From
  my research I found that there is a way to do this via the REST API, but
  it isn't documented for some reason. Doing a POST to /{session
  id}/jobs/{job id}/cancel will cancel a job. As for the Java API, the
  feature isn't part of the Java interface, but most implementations of it
  add it, such as the Scala API which ScalaJobHandle class on sumbit which
  has a cancel function. I'm not sure how you're submitting you jobs, but
  there should be a cancel function available to you somewhere depending on
  the client you're using. From this discussion I've realized our current
  documentation is even more lacking that I had thought.



                                                                                   
 Alex Bozarth                                                                      
 Software Engineer                                                                 
 Spark Technology Center                                                           
                                                                                   


                                                                                    
                                                                                    
                                                                                    
 E-mail: ajbozart@us.ibm.com                                                        
 GitHub: github.com/ajbozarth                                                       
                                                                  505 Howard Street 
                                                            San Francisco, CA 94105 
                                                                      United States 
                                                                                    





  Inactive hide details for kant kodali ---01/18/2018 06:09:59 PM---Also
  just tried the below and got the state. It ended up in "kant kodali
  ---01/18/2018 06:09:59 PM---Also just tried the below and got the state.
  It ended up in "FAILED" stated when I expected it to be

  From: kant kodali <kanth909@gmail.com>
  To: user@livy.incubator.apache.org
  Date: 01/18/2018 06:09 PM
  Subject: Re: How to cancel the running streaming job using livy?




  Also just tried the below and got the state. It ended up in "FAILED"
  stated when I expected it to be in "CANCELLED" state. Also from the docs
  it is not clear if it kills the session or the job? if it kills the
  session I can't spawn any other Job. Sorry cancelling job had been a bit
  confusing for me.
  DELETE /sessions/0



  On Thu, Jan 18, 2018 at 5:55 PM, kant kodali <kanth909@gmail.com> wrote:
        oh this raises couple questions.

        1) Is there a programmatic way to cancel a job?

        2) isĀ  there any programmatic way to get session id? If not, how do
        I get a sessionId when I spawn multiple jobs or multiple sessions?


        On Thu, Jan 18, 2018 at 5:39 PM, Alex Bozarth <ajbozart@us.ibm.com>
        wrote:
        You make a DELETE call as detailed here:
        http://livy.apache.org/docs/latest/rest-api.html#response

                                                                                   
 Alex Bozarth                                                                      
 Software Engineer                                                                 
 Spark Technology Center                                                           
                                                                                   


                                                                                    
                                                                                    
                                                                                    
 E-mail: ajbozart@us.ibm.com                                                        
 GitHub: github.com/ajbozarth                                                       
                                                                  505 Howard Street 
                                                            San Francisco, CA 94105 
                                                                      United States 
                                                                                    





        Inactive hide details for kant kodali ---01/18/2018 05:34:07
        PM---Hi All, I was able to submit a streaming job to livy however
        kant kodali ---01/18/2018 05:34:07 PM---Hi All, I was able to
        submit a streaming job to livy however I wasn't able to find

        From: kant kodali <kanth909@gmail.com>
        To: user@livy.incubator.apache.org
        Date: 01/18/2018 05:34 PM
        Subject: How to cancel the running streaming job using livy?



        Hi All,

        I was able to submit a streaming job to livy however I wasn't able
        to find any way to cancel the running the job? Please let me know.

        Thanks!









--
Marcelo


Mime
View raw message