hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Robertson (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-6111) CLONE - Map tasks not local to RS
Date Sun, 27 May 2012 08:10:22 GMT

     [ https://issues.apache.org/jira/browse/HBASE-6111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tim Robertson updated HBASE-6111:
---------------------------------

          Description: 
I have started seeing this issue in our environment.  HBASE-1672 was closed as non reproducible,
so I cloned it here.

I have a 367M record table, compressed with snappy, and running a vanilla MR SCAN with no
filters spawns 441 Mappers.  The cluster has 216 slots for mappers, and the first wave all
report 100% data-local mappers.  As the second wave of mappers come up they don't get run
locally to the RS and data locality drops.

This kills our environment, as it saturates the network at 120M which is very clear on ganglia.

I am really happy to help diagnose this, but need some guidance on what to do.  I don't know
enough yet about how task assignment works in MR to determine why the machines are picking
up random tasks for their second effort and not one for the local RS.

  was:
The number of data local map tasks while scanning a table is only about 10% of the total map
tasks...
My table had 280 regions and 13M records... The number of map tasks in the scan job were equal
to the number of regions (280). Only 25 of them were data local tasks.

          Environment: DN, TT and RS running on the same nodes, all using CDH3.  Ganglia monitoring
everything.  (was: DN, TT and RS running on the same nodes.)
    Affects Version/s:     (was: 0.19.3)
                           (was: 0.20.0)
                       0.20.2
                       0.92.0
        Fix Version/s:     (was: 0.20.0)
    
> CLONE - Map tasks not local to RS
> ---------------------------------
>
>                 Key: HBASE-6111
>                 URL: https://issues.apache.org/jira/browse/HBASE-6111
>             Project: HBase
>          Issue Type: Bug
>          Components: mapred, master, regionserver
>    Affects Versions: 0.20.2, 0.92.0
>         Environment: DN, TT and RS running on the same nodes, all using CDH3.  Ganglia
monitoring everything.
>            Reporter: Tim Robertson
>
> I have started seeing this issue in our environment.  HBASE-1672 was closed as non reproducible,
so I cloned it here.
> I have a 367M record table, compressed with snappy, and running a vanilla MR SCAN with
no filters spawns 441 Mappers.  The cluster has 216 slots for mappers, and the first wave
all report 100% data-local mappers.  As the second wave of mappers come up they don't get
run locally to the RS and data locality drops.
> This kills our environment, as it saturates the network at 120M which is very clear on
ganglia.
> I am really happy to help diagnose this, but need some guidance on what to do.  I don't
know enough yet about how task assignment works in MR to determine why the machines are picking
up random tasks for their second effort and not one for the local RS.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message