chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jie Huang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CHUKWA-647) Spread out intermediate data with the same ReduceType into different Reduce Tasks
Date Mon, 16 Jul 2012 05:29:33 GMT
Jie Huang created CHUKWA-647:
--------------------------------

             Summary: Spread out intermediate data with the same ReduceType into different
Reduce Tasks
                 Key: CHUKWA-647
                 URL: https://issues.apache.org/jira/browse/CHUKWA-647
             Project: Chukwa
          Issue Type: Improvement
          Components: Data Processors
    Affects Versions: 0.4.0, 0.6.0
            Reporter: Jie Huang
            Priority: Minor


We have found that if we partitioned the map output data according to ReduceType, we can see
the data skew in some HiTune cases. Then one or two Reduce Tasks slow down the whole Demux
job somehow, since those reduce tasks have to process more input-data.    

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message