hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5870) Support for passing Job priority through Application Submission Context in Mapreduce Side
Date Wed, 19 Aug 2015 13:42:46 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14703032#comment-14703032
] 

Sunil G commented on MAPREDUCE-5870:
------------------------------------

Thanks [~jlowe] for sharing the inputs.
Yes, as you mentioned it wont be as perfect as we cant map some undefined numerical constants
to mapred defined enums.
Few points:
- As you have mentioned, we could add a DEFAULT enum to indicate a case where no priority
is set from client side. But now YARN has the capability to take default value, if priority
is not set from client side. This can be Queue default or Cluster default (if Queue doesnt
have any default or queue doesnt exist). And mapred cannot know this. So could we add a DEFAULT
enum with 0 as base value for now.
- There is another problem when we do integer mapping. 
{code}
public enum JobPriority {
  VERY_HIGH,
  HIGH,
  NORMAL,
  LOW,
  VERY_LOW;
}
{code}
We wont be able to take *ordinal* here because of the current order. Hence {TypeConver#toYarn}}
can help in mapping this enum to a YARN model of integer priority (preferably from 1 to 5).
So I can make 0 as DEFAULT (new enum)
- I agree with your idea of having a new enum which can be returned when any other integer
is been used a priority other than standard enum. Eventhough its not much helpful as its not
giving exact priority value,  I feel we can also have a new api {{public int getJobPriorityAsInteger}}
along with existing one for more clarity in such cases.


> Support for passing Job priority through Application Submission Context in Mapreduce
Side
> -----------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5870
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5870
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-MAPREDUCE-5870.patch, 0002-MAPREDUCE-5870.patch, 0003-MAPREDUCE-5870.patch,
Yarn-2002.1.patch
>
>
> Job Prioirty can be set from client side as below [Configuration and api].
> 			a.	JobConf.getJobPriority() and Job.setPriority(JobPriority priority) 
> 			b.	We can also use configuration "mapreduce.job.priority".
> 		Now this Job priority can be passed in Application Submission context from Client side.
> 		Here we can reuse the MRJobConfig.PRIORITY configuration. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message