hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mapred Learn <mapred.le...@gmail.com>
Subject Re: Moving Files to Distributed Cache in MapReduce
Date Fri, 29 Jul 2011 18:18:45 GMT
I hope my previous reply helps...

On Fri, Jul 29, 2011 at 11:11 AM, Roger Chen <rogchen@ucdavis.edu> wrote:

> After moving it to the distributed cache, how would I call it within my
> MapReduce program?
>
> On Fri, Jul 29, 2011 at 11:09 AM, Mapred Learn <mapred.learn@gmail.com
> >wrote:
>
> > Did you try using -files option in your hadoop jar command as:
> >
> > /usr/bin/hadoop jar <jar name> <main class name> -files  <absolute
path
> of
> > file to be added to distributed cache> <input dir> <output dir>
> >
> >
> > On Fri, Jul 29, 2011 at 11:05 AM, Roger Chen <rogchen@ucdavis.edu>
> wrote:
> >
> > > Slight modification: I now know how to add files to the distributed
> file
> > > cache, which can be done via this command placed in the main or run
> > class:
> > >
> > >        DistributedCache.addCacheFile(new
> URI("/user/hadoop/thefile.dat"),
> > > conf);
> > >
> > > However I am still having trouble locating the file in the distributed
> > > cache. *How do I call the file path of thefile.dat in the distributed
> > cache
> > > as a string?* I am using Hadoop 0.20.2
> > >
> > >
> > > On Fri, Jul 29, 2011 at 10:26 AM, Roger Chen <rogchen@ucdavis.edu>
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Does anybody have examples of how one moves files from the local
> > > > filestructure/HDFS to the distributed cache in MapReduce? A Google
> > search
> > > > turned up examples in Pig but not MR.
> > > >
> > > > --
> > > > Roger Chen
> > > > UC Davis Genome Center
> > > >
> > >
> > >
> > >
> > > --
> > > Roger Chen
> > > UC Davis Genome Center
> > >
> >
>
>
>
> --
> Roger Chen
> UC Davis Genome Center
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message