hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Halperin <lomax0...@gmail.com>
Subject Re: Job scheduling
Date Tue, 07 Jun 2011 20:00:36 GMT
I found from Googling around that I should probably be seeing messages like
"Choosing data-local task" and "Choosing rack-local task" - from
JobInProgress::addRunningTaskToTIP(). (e.g. here:
http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-dev/201012.mbox/%3C2120373776.44711293532894724.JavaMail.hudson@aegis%3E).

I'm digging through the code, but any idea why mine's always choosing
non-local trackers?


On 7 June 2011 12:41, Ian Halperin <lomax0000@gmail.com> wrote:

> Harsh, thanks for the clarification.
>
> But my mappers always seem to run elsewhere. Here's an example with 2
> splits, both on rack1node1, but the 2 mappers get started on other nodes.
> Could the "choosing a non-local task" message be significant?
>
> I have actually read through the JobTracker source, but didn't get too far
> in understanding the logic. :)
>
> The only other odd-looking thing is that trackers have "localhost/
> 127.0.0.1" in their names as well as their real hostnames - not sure if
> that could be related.
>
> ...
> mapred.JobClient: Running job: job_201106071125_0004
> org.apache.hadoop.mapred.JobInProgress: jobToken generated and stored with
> users keys in /mapred/system/job_201106071125_0004/jobToken
> org.apache.hadoop.mapred.JobInProgress: Input size for job
> job_201106071125_0004 = 100086130. Number of splits = 2
> org.apache.hadoop.mapred.JobInProgress: tip:task_201106071125_0004_m_000000
> has split on node:/rack1/rack1node1.local
> org.apache.hadoop.mapred.JobInProgress: tip:task_201106071125_0004_m_000001
> has split on node:/rack1/rack1node1.local
> org.apache.hadoop.mapred.JobInProgress: job_201106071125_0004
> LOCALITY_WAIT_FACTOR=0.16666667
> org.apache.hadoop.mapred.JobInProgress: Job job_201106071125_0004
> initialized successfully with 2 map tasks and 1 reduce tasks.
> org.apache.hadoop.mapred.JobTracker: Adding task (JOB_SETUP)
> 'attempt_201106071125_0004_m_000003_0' to tip
> task_201106071125_0004_m_000003, for tracker
> 'tracker_rack3node1.local:localhost/127.0.0.1:55842'
> mapred.JobClient:  map 0% reduce 0%
> org.apache.hadoop.mapred.JobInProgress: Task
> 'attempt_201106071125_0004_m_000003_0' has completed
> task_201106071125_0004_m_000003 successfully.
> org.apache.hadoop.mapred.JobInProgress: Choosing a non-local task
> task_201106071125_0004_m_000000
> org.apache.hadoop.mapred.JobTracker: Adding task (MAP)
> 'attempt_201106071125_0004_m_000000_0' to tip
> task_201106071125_0004_m_000000, for tracker
> 'tracker_rack3node1.local:localhost/127.0.0.1:55842'
> org.apache.hadoop.mapred.JobInProgress: Choosing a non-local task
> task_201106071125_0004_m_000001
> org.apache.hadoop.mapred.JobTracker: Adding task (MAP)
> 'attempt_201106071125_0004_m_000001_0' to tip
> task_201106071125_0004_m_000001, for tracker
> 'tracker_rack2node1.local:localhost/127.0.0.1:37910'
> ...
>
>
> On 7 June 2011 04:59, Harsh J <harsh@cloudera.com> wrote:
>
>> Ian,
>>
>> The reducer needn't run on the same TaskTracker as it does not work on
>> local data. The mapper 000021, however, should've run on
>> node:/rack1/rack1node1.local.
>>
>> On Tue, Jun 7, 2011 at 5:09 AM, Ian Halperin <lomax0000@gmail.com> wrote:
>> > Hi,
>> >
>> > I might be misunderstanding how scheduling is supposed to work, or I
>> might
>> > have something misconfigured, but my Map/Reduce jobs don't seem to run
>> where
>> > my data is located.
>> >
>> > I get a bunch of these messages:
>> > INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201106062049_0001_
>> > m_000021 has split on node:/rack1/rack1node1.local
>> > ... indicating it has correctly found the source data at my node
>> > /rack1/rack1node1 (the only copy of the data - for the purpose of this
>> > experiment I have set dfs.replication = dfs.replication.min =
>> > dfs.replication.max = 1 so I only have 1 replica).
>> >
>> > However, it then goes on to run the JOB_SETUP, MAP, REDUCE, JOB_CLEANUP
>> > tasks on abitrary tasktrackers, usually not where the data is located,
>> so
>> > the first thing they have to do is pull it over the network from another
>> > node.
>> >
>> > Did I miss something - or hopefully configure something wrong? :)
>> >
>> > Ian
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>

Mime
View raw message