hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits
Date Tue, 02 Apr 2013 07:09:17 GMT

    [ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619580#comment-13619580

Hitesh Shah commented on YARN-193:

+    and will get capped to this value. When it is set to -1, checking against the
+    maximum allocation should be disable.</description>

I am not sure if we should allow disabling of the max memory and max vcores setting. Was it
supported earlier or does the patch introduce this support?

Spelling mistake: alloacated

+        LOG.info("Resource request was not able to be alloacated for" +
+            " application attempt " + appAttemptId + " because it" +
+            " failed to pass the validation. " + e.getMessage());

The above could be made more simple and brief. For example, "LOG.warn("Invalid resource ask
by application " + appAttemptId, e);" . Also, please use LOG.level(message, throwable) when
trying to log an exception. 

+        RPCUtil.getRemoteException(e);

Above is missing a throw. 

Likewise, in handling of submitApplication, please change log level to warn and also use the
correct log function instead of using e.getMessage(). 

     if (globalMaxAppAttempts <= 0) {
       throw new YarnException(
           "The global max attempts should be a positive integer.");

Unrelated to this patch but when throwing/logging errors related to configs, we should always
point to the configuration property to let the user know which property needs to be changed.
Please file a separate jira for the above. With respect to this, it may be useful to point
to the property when throwing exceptions for invalid min/max memory/vcores.

Unnecessary import in RMAppAttemptImpl:

 +import org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils;

For InvalidResourceRequestException, missing javadocs for class description. 

Question - should normalization of resource requests be done inside the scheduler or in the
ApplicationMasterService itself which handles the allocate call?

If maxMemory or maxVcores is set to -1, what will happen when normalize() is called? I think
there are missing tests related to use of DISABLE_RESOURCELIMIT_CHECK in both validate and
normalize functions that should have caught this error. In any case, the main question is
whether DISABLE_RESOURCELIMIT_CHECK should actually be allowed.

> Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation
> ---------------------------------------------------------------------------------------------------------
>                 Key: YARN-193
>                 URL: https://issues.apache.org/jira/browse/YARN-193
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.0.2-alpha, 3.0.0
>            Reporter: Hitesh Shah
>            Assignee: Zhijie Shen
>         Attachments: MR-3796.1.patch, MR-3796.2.patch, MR-3796.3.patch, MR-3796.wip.patch,
YARN-193.10.patch, YARN-193.11.patch, YARN-193.4.patch, YARN-193.5.patch, YARN-193.6.patch,
YARN-193.7.patch, YARN-193.8.patch, YARN-193.9.patch

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message