hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Grover <mgro...@oanda.com>
Subject Re: Accessing distributed cache in transform scripts
Date Wed, 19 Oct 2011 13:46:01 GMT
Mohit,
I use Hive 0.7.1 and am able to access the file from distributed cache just by filename. Did
you try that?

Mark

----- Original Message -----
From: "Chinna Rao Lalam 72745" <chinnarao@huawei.com>
To: user@hive.apache.org
Sent: Wednesday, October 19, 2011 6:56:38 AM
Subject: Re: Accessing distributed cache in transform scripts

Hi,

  Can u post some more details like for the  "list file"  what command u have used.


----- Original Message -----
From: Mohit Gupta <success.mohit.gupta@gmail.com>
Date: Wednesday, October 19, 2011 3:16 pm
Subject: Re: Accessing distributed cache in transform scripts
To: user@hive.apache.org

> Plz help...Any pointers!!
> 
> On 10/19/11, Mohit Gupta <success.mohit.gupta@gmail.com> wrote:
> > Hi All,
> >
> > I want some read-only data to be available at the reducers / 
> transform> scripts. I am trying to use distributed cache to 
> achieve this using
> > the following steps:
> > 1.  add file s3://bucket_name/prefix/testfile
> > then
> > 2.  "list file" to find out the location of local copy of testfile.
> > it shows the locally file is stored at
> > "/mnt/var/lib/hive_05/downloaded_resources/testfile"
> >
> > 3. reduce script code ( in php)
> > #!/usr/bin/php
> > <?php
> > 
> $fin=fopen("/mnt/var/lib/hive_05/downloaded_resources/testfile","r");> $line=fgets($fin);
> > ?>
> >
> > 4. the reduce script throws an error "no such file or directory
> > /mnt/var/lib/hive_05/downloaded_resources/testfile ".
> >
> > 5. On ssh, I found that the file was indeed present at
> > "/mnt/var/lib/hive_05/downloaded_resources/testfile".
> >
> > Plz help. Is this the right way to pass some info to the reduce
> > scripts. If yes, what I could be missing here? Else, any other
> > possible way(s)!!
> >
> > Thanks
> >
> > Mohit
> >
> 
> 
> -- 
> Best Regards,
> 
> Mohit Gupta
> Software Engineer at Vdopia Inc.
> 

Mime
View raw message