hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6509) Add a size threshold beyond which yarn logs will require a force option
Date Mon, 01 May 2017 18:57:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991305#comment-15991305

Junping Du commented on YARN-6509:

bq. Minor point of correction, 2^63 -1 is something like 8.3 million TiB.
My bad. You are right, it is ~8 EB which should be large enough.

bq. I'd like it if we could find some operational users to weigh in on wether 10GiB is enough
for a default. could we ask on the yarn user list?
I doubt we can have a uniformed value for different users that operated on different scale
of cluster. My personal feeling is over 10GB log file is hard to analysis but it happen often
for large job on some large clusters. Or we can keep above size as default which almost means
no limit by default and let each user/admin set this value case by case if they want to?‚Äč

> Add a size threshold beyond which yarn logs will require a force option
> -----------------------------------------------------------------------
>                 Key: YARN-6509
>                 URL: https://issues.apache.org/jira/browse/YARN-6509
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Siddharth Seth
>            Assignee: Xuan Gong
>             Fix For: 2.9.0
>         Attachments: YARN-6509.1.patch
> An accidental fetch for a long running application can lead to scenario which the large
size of log can fill up a disk.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message