hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Lewis <lordjoe2...@gmail.com>
Subject Problems accessing the local file system on AWS MapReduce
Date Tue, 03 Aug 2010 15:44:43 GMT
I am trying to use Distributes Cache on AWS Elastic Map educe

Using S3Fox I see I have two files in my bucket lordjoe

In My main I say
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
String[] otherArgs = new GenericOptionsParser(conf,
Job job = new Job(conf, "WholeFiles");
Path("s3n://lordjoe/Hello#Hello.x").toUri(), conf);
Path("s3n://lordjoe/FeeFie.txt#FeeFie.txt").toUri(), conf);
boolean ans = job.waitForCompletion(true);

In my reducer I say
protected void reduce(final KEYIN key, final Iterable<VALUEIN> values, final
Context context) throws IOException, InterruptedException {
Configuration configuration = context.getConfiguration();
Path[] files = DistributedCache.getLocalCacheFiles(configuration);
if (files != null) {
for (int i = 0; i < files.length; i++) {
Path file = files *" *
Steven M. Lewis PhD
Institute for Systems Biology
Seattle WA

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message