hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From felix gao <gre1...@gmail.com>
Subject How do hadoop work in details
Date Wed, 29 Dec 2010 22:43:02 GMT
Hi all,

I am trying to figure out how exactly happens inside the job.

1) When the jobtracker launches a task to be run, how does it impact the
currently running jobs if the the current running job have higher, same, or
lower priories using the default queue.

2) What if a low priority job is running that is holding all the reducer
slots and the mappers are halfway done and a high priority job comes in took
all the mappers but cannot complete but all the reducer slots are taken by
the low priority job?

3) when is mappers allocated on the slaves, and when is reducers allocated.

4)Does mappers pass all the data to reducers using RPC or they write their
output to HDFS and the reducers pick it up.

5) within a job, when and where is all the io occurs.

I know this seems to be a lot of low level questions , if you can point me
to the right place to look is should be enough.



View raw message