livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vivek Suvarna <vikk...@gmail.com>
Subject Re: Input file as an argument og a Spark code
Date Tue, 25 Jul 2017 05:18:52 GMT
I had a similar requirement. 
I used webhdfs to first copy the file across to hdfs before starting the spark job via Livy.



Sent from my iPhone

> On 25 Jul 2017, at 9:39 AM, Saisai Shao <sai.sai.shao@gmail.com> wrote:
> 
> I think you have to make this csv file accessible from Spark cluster, putting to HDFS
is one possible solution. 
> 
>> On Tue, Jul 25, 2017 at 1:26 AM, Joaquín Silva <joaquin@akainix.com> wrote:
>> Hello,
>> 
>>  
>> 
>> I'm building a BASH program (using Curl)  that should run a Spark code remotely using
Livy. But one of the code argument  is a CSV file, how can I make that spark reads this file?.
The file is going to be in client side, not in the Spark cluster machines.
>> 
>>  
>> 
>> Regards,
>> 
>>  
>> 
>> Joaquín Silva
>> 
>>  
>> 
> 

Mime
View raw message