livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Bozarth" <ajboz...@us.ibm.com>
Subject Re: POST /batches failing with "Only local python files are supported"
Date Thu, 02 Nov 2017 21:34:13 GMT
It sounds like you're passing in a local past and it's being treated as an
HDFS path. Have you tired passing the path in with file:// at the front
(similar to hdfs://) that tells hdfs that the path is local, I've run into
this issue with Spark before.

                                                                                   
 Alex Bozarth                                                                      
 Software Engineer                                                                 
 Spark Technology Center                                                           
                                                                                   
                                                                                     
                                                                                     
                                                                                     
 E-mail: ajbozart@us.ibm.com                                                         
 GitHub: github.com/ajbozarth                                                        
                                                                   505 Howard Street 
                                                             San Francisco, CA 94105 
                                                                       United States 
                                                                                     








From:	"Partridge, Lucas (GE Aviation)" <Lucas.Partridge@ge.com>
To:	"user@livy.incubator.apache.org"
            <user@livy.incubator.apache.org>
Date:	11/02/2017 07:14 AM
Subject:	POST /batches failing with "Only local python files are
            supported"



I want to use Livy (0.3) to run a Python file that I’ve placed in HDFS.
I’m invoking POST /batches from a Java REST client, passing in the path to
the HDFS file as the ‘file’ argument of the POST request’s body (
https://github.com/apache/incubator-livy/tree/branch-0.3#request-body-2).
The value I’m providing for ‘file’ is "/user/MyUserName/hello.py".

The POST response says the batch is in state ‘starting’ but when I query it
using GET /batches/{batchId} I see this:

Error: Only local python files are supported:
hdfs://NNPRDHA/user/MyUserName/hello.py

You can see the value has been altered from what I provided. How do I
successfully invoke the Python file please?

(I see someone raised a similar problem at
https://groups.google.com/a/cloudera.org/d/msg/livy-user/6AZeqtVwipg/U46tUjqNBwAJ
 but it’s not clear if or how they solved it.)

Thanks, Lucas.

Mime
View raw message