hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject difference between add file from a local disk and hdfs file
Date Sat, 16 May 2015 21:16:16 GMT
Hi,

I am trying to access a lookup file from a udf.
There are two ways I add lookup file to distribute cache

option1: loading file from local disk to distributed cache this is for hive
cli
add file tmp.txt;


option2: add a file from hdfs to distributed cache so that oozie can do it
too
add file hdfs:///user/admin/tmp.txt;


i want to use a file from hdfs into distributed cache so that I can use it
a hive udf.

Problem is
when I load a file using option1. it is available to the udf (works fine)
hive> add file format.txt;
Added resources: [format.txt]
hive> list files;
format.txt


But when I load the file from hdfs, it moves into tmp folder  and i am not
sure if the path remains same all the time
hive> add file hdfs:////user/admin/tmp.txt ;
converting to local hdfs:///user/admin/tmp.txt
Added resources: [hdfs:////tmp.txt]
hive> list files;

/tmp/006ab981-ddac-4bcb-bee1-7d8ed9a271a0_resources/tmp.txt

Question:  how do I get the file at same location (like option 1 all times)
 cause from option 2 I keep getting the error tmp.txt does not exists when
I initialize the udf

thanks
-- 
Nitin Pawar

Mime
View raw message