hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <>
Subject difference between add file from a local disk and hdfs file
Date Sat, 16 May 2015 21:16:16 GMT

I am trying to access a lookup file from a udf.
There are two ways I add lookup file to distribute cache

option1: loading file from local disk to distributed cache this is for hive
add file tmp.txt;

option2: add a file from hdfs to distributed cache so that oozie can do it
add file hdfs:///user/admin/tmp.txt;

i want to use a file from hdfs into distributed cache so that I can use it
a hive udf.

Problem is
when I load a file using option1. it is available to the udf (works fine)
hive> add file format.txt;
Added resources: [format.txt]
hive> list files;

But when I load the file from hdfs, it moves into tmp folder  and i am not
sure if the path remains same all the time
hive> add file hdfs:////user/admin/tmp.txt ;
converting to local hdfs:///user/admin/tmp.txt
Added resources: [hdfs:////tmp.txt]
hive> list files;


Question:  how do I get the file at same location (like option 1 all times)
 cause from option 2 I keep getting the error tmp.txt does not exists when
I initialize the udf

Nitin Pawar

View raw message