hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6509) Add a size threshold beyond which yarn logs will require a force option
Date Wed, 27 Sep 2017 23:58:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183474#comment-16183474
] 

Wangda Tan commented on YARN-6509:
----------------------------------

[~xgong],  

Thanks for the patch, several comments regarding to CLI options: 
- Existing unit of log size limit is bytes, do you think we should use "MB" as minimum unit
so user doesn't need to find a calculator to get the value? If you agree so, it's better to
update "size_limit" to "size_limit_mb".
- We may not need a separate ignore_size_limit, just specify -1 to size_limit to disable size
limit should be good enough. 

For help message: 
{code}
868	    opts.addOption(SIZE_LIMIT_OPTION, true, "Use this option to limit "
869	        + "the size of the total logs which could be fetched. "
870	        + "By default, the value is 10G.");
871	    opts.addOption(IGNORE_SIZE_LIMIT_OPTION, false,
872	        "Use this option to ignore the total log size limit. By default, "
873	        + "we only allow to fetch at most 10G logs. If the total log size is "
874	        + "larger than 10G, the CLI would fail. The user could specify this "
875	        + "option to ignore the size limit and fetch all logs.");
{code}

- "10G" should not be hard coded, it's better to use LOG_SIZE_LIMIT_DEFAULT. 
- Also "we only allow to fetch at most 10G logs" should ref to size_limit.

> Add a size threshold beyond which yarn logs will require a force option
> -----------------------------------------------------------------------
>
>                 Key: YARN-6509
>                 URL: https://issues.apache.org/jira/browse/YARN-6509
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Assignee: Xuan Gong
>             Fix For: 2.9.0
>
>         Attachments: YARN-6509.1.patch, YARN-6509.2.patch, YARN-6509.3.patch, YARN-6509.4.patch
>
>
> An accidental fetch for a long running application can lead to scenario which the large
size of log can fill up a disk.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message