hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-2548) Find a more scalable way to handle logs for long running service
Date Fri, 12 Sep 2014 19:57:35 GMT
Xuan Gong created YARN-2548:
-------------------------------

             Summary: Find a more scalable way to handle logs for long running service
                 Key: YARN-2548
                 URL: https://issues.apache.org/jira/browse/YARN-2548
             Project: Hadoop YARN
          Issue Type: Sub-task
            Reporter: Xuan Gong
            Assignee: Xuan Gong


After YARN-2468, the container logs will be aggregated separately based on the time. It will
increate the total number of log files. It is fine for small cluster. But for the larger cluster,
it will make too-many-files problem even worse.

We need to find a more scalable way to handle those logs. Aggregate all container logs in
a single file is an option, but we need to find a different way, other than TFile(do not support
append), to do it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message