hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lohit.vijayar...@yahoo.com
Subject Re: can jobs be launched recursively within a mapper ?
Date Mon, 29 Oct 2007 21:00:16 GMT
>Can I recursively launch a job within the map() method of a mapper?

Interesting. If your launch is blocking, then I am thinking possibility of deadlock. Lets
say you have N nodes and your initial map tasks are scheduled on all N nodes. Now if each
of the map tasks launch few more jobs, then the new ones are sitting there waiting for free
nodes.

Thanks,
Lohit
----- Original Message ----
From: Jim the Standing Bear <standingbear@gmail.com>
To: hadoop-user@lucene.apache.org
Sent: Monday, October 29, 2007 1:28:17 PM
Subject: can jobs be launched recursively within a mapper ?


Hi,

Can I recursively launch a job within the map() method of a mapper?
The task i am facing involves extracting metadata for filenames and
catalog names from hierarchical catalogs, and for each catalog name
extract, I must recursively do the same thing until no more catalog
names present.  Therefore, it will be convenient if I can launch a job
within the map(), but I want to make certain that it will work before
investing too much efforts into the coding.

If it won't work, could you suggest the proper way to go about doing
so?  Much appreciated in advance.

-- Jim




__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Mime
View raw message