hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Castagna <castagna.li...@googlemail.com>
Subject Sequence of MapReduce jobs... is it possible to upload the jar only once?
Date Tue, 28 Jun 2011 16:42:31 GMT
Hi,
I have a sequence of three MapReduce jobs which I run this way:

  FirstDriver first = new FirstDriver(configuration);
  first.run(new String[] { args[0], args[1] + "_1" });
  SecondDriver second = new SecondDriver(configuration);
  second.run(new String[] { args[1] + "_1", args[1] + "_2" });
  ThirdDriver third = new ThirdDriver(configuration);
  third.run(new String[] { args[1] + "_2", args[1] + "_3" });

Each {First|Second|Third}Driver extends Configured and implements Tool.

My jar has quite a few dependencies, is there a way to upload to HDFS
the jar only once?

Thank you,
Paolo

Mime
View raw message