hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "rahul k singh (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1594) Support for Sleep Jobs in gridmix
Date Tue, 06 Apr 2010 05:31:33 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853759#action_12853759

rahul k singh commented on MAPREDUCE-1594:

I have implemented all the comments except few:

- GenerateData should extend from GridmixJob instead of LoadJob. I think we can have a default
implementation of buildSplits (as an empty function) in GridmixJob and remove the "abstract"

GenerateData is now extending GridmixJob. But GridmixJob is still abstract as call() method
is abstract.And it is implemented by all the derived classes.

- Avoiding directly setting "gridmix.job.seq" in both LoadJob and SleepJob. Instead, refactor
the statement to a common method in GridmixJob called setSeqId(Job job). Similarly, adding
a method getSeqId(Job job) in GridmixJob and avoid directly calling conf.get("girdmix.job.seq",
-1) in {GridmixInputFormat, SleepInputFormat}.getSplits(...).

getSeqId is not there as {GridmixInputFormat, SleepInputFormat}.getSplits(...). is part of
static inner classes and can only access static method. 

- I cannot find anywhere outdir is used by SleepJob. Did you encounter an error if FOF.setOutputPath
is commented out in SleepJob.call()?
Removed this code and tested , things work fine.

- Both SleepJob and GridmixJob calls FileInputFormat.addInputPath(job, new path("ignored")),
but one is surrounded with a try-catch block and the other is not. Not sure why. I am also
curious to know what would be the error if FIF.addInputPath is not called in both classes

I have remove FIF.addInputPath . things are working fine on cluster. I have removed try-catch
block and added the exception in the signature of call

> Support for Sleep Jobs in gridmix
> ---------------------------------
>                 Key: MAPREDUCE-1594
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1594
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: contrib/gridmix
>            Reporter: rahul k singh
>         Attachments: 1376-5-yhadoop20-100-3.patch, 1594-yhadoop-20-1xx-1-2.patch, 1594-yhadoop-20-1xx-1-3.patch,
1594-yhadoop-20-1xx-1-4.patch, 1594-yhadoop-20-1xx-1.patch, 1594-yhadoop-20-1xx.patch
> Support for Sleep jobs in gridmix

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message