hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "WANG Shicai" <Evan...@yahoo.cn>
Subject Need Suggestion: Tuning MR performance by changing parameters in Hadoop project and JVM
Date Wed, 02 Jun 2010 07:52:55 GMT
Hi,

This message is a little long. So I beg your patient.

Our team would like to tune MR performance by changing parameters in Hadoop project and JVM
according to the MR Job status and result.

First, classify MR jobs into several kinds. Then monitor cpu, memory, etc. in a MR job, structing
the data from the monitor and input it into HBase. The crucial step is to build a model or
models to analysis the data. Finally, acquire the proposal for tuning MR jobs, such as increase
the memory for the job or reduce it, etc.

However, I am a developer in HBase subproject and not so acquainted with MR jobs. I need some
suggestion about the following aspects:

* Is this plan feasible or not? why?
* Is there any one or team doing the above before?
* Which processes in a MR job we ought to monitor more carefully?
* Which parameters in that processes we ought to care?
* What can we refer for the model building?
* Also, any other suggestion about our plan will be welcome.

Thank you a lot!!!

Evan,
2010-06-02

__________________________________________________
ϿעŻ?
http://cn.mail.yahoo.com


Mime
View raw message