hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <o...@yahoo-inc.com>
Subject Re: How do I run job on the every nodes without input file.
Date Mon, 02 Apr 2007 18:22:48 GMT

On Apr 2, 2007, at 1:48 AM, 김형준 wrote:

> Generally, map task runs on a special node by FileSplit.getSplit().
> In my case, I have no input file. and I want to run task on the  
> every nodes
> in hadoop cluster.
> It's like LSF or other job queue.
> How do I run job on the every node without input file?

There are a couple of approaches that would work. One example is  
RandomWriter, which takes no input and just writes a set of random  
data files. It defines an InputFormat that generates the requested  
number of splits (and therefore maps) and creates a string for each  
and each one is given a single record with the generated string.

-- Owen
View raw message