hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay Kumar Thota (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2192) Gridmix tests for MR streaming job traces.
Date Thu, 18 Nov 2010 15:03:15 GMT
Gridmix tests for MR streaming job traces.
------------------------------------------

                 Key: MAPREDUCE-2192
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2192
             Project: Hadoop Map/Reduce
          Issue Type: Task
          Components: contrib/gridmix
            Reporter: Vinay Kumar Thota
            Assignee: Vinay Kumar Thota


1. Generate input data based on cluster size and create the synthetic jobs by using the 2
min folded MR streaming jobs trace and submit the jobs with below arguments.
GRIDMIX_JOB_TYPE = LOADJOB
GRIDMIX_USER_RESOLVER = SubmitterUserResolver
GRIDMIX_SUBMISSION_POLICY = STRESS
GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = True
Input Size = 150 MB * No. of nodes in cluster.
GRIDMIX_KEY_FRC = 0.5f
TRACE_FILE = 2 min folded trace.
Verify JobStatus for each job, input split size for each job and summary (QueueName, UserName,
StatTime, FinishTime, maps, reducers and counters etc) after completion of execution.

2.  Generate input data based on cluster size and create the synthetic jobs by using the 3
min folded MR streaming jobs trace and submit the jobs with below arguments.
GRIDMIX_JOB_TYPE = LoadJob
GRIDMIX_USER_RESOLVER = RoundRobinUserResolver
GRIDMIX_MIN_FILE = 150 MB
GRIDMIX_SUBMISSION_POLICY = STRESS
GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = True
BYTES_PER_FILE = 200 MB
Input Size = 250 MB * No. of nodes in cluster.
PROXY_USERS = proxy users file path
TRACE_FILE = 3 min folded trace.
Verify JobStatus for each job, input split size for each job and summary (QueueName, UserName,
StatTime, FinishTime, maps, reducers and counters etc) after completion of execution.

3. Generate input data based on cluster size and create the synthetic jobs by using the 5
min MR High RAM jobs trace and submit the jobs with below arguments.
GRIDMIX_JOB_TYPE = LoadJob
GRIDMIX_USER_RESOLVER = SubmitterUserResolver
GRIDMIX_SUBMISSION_POLICY = Replay
GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
BYTES_PER_FILE = 200 MB
Input Size = 500 MB * No. of nodes in cluster.
TRACE_FILE = 5 min folded trace.
Verify JobStatus for each job and summary (QueueName, UserName, StatTime, FinishTime, MAPS,
REDUCERS and COUNTERS etc) after completion of execution.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message