reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Weimer (JIRA)" <j...@apache.org>
Subject [jira] [Created] (REEF-1837) IsParentAliveCancellationDetector logs lack of HTTP endpoint excessively
Date Mon, 24 Jul 2017 15:55:00 GMT
Markus Weimer created REEF-1837:
-----------------------------------

             Summary: IsParentAliveCancellationDetector logs lack of HTTP endpoint excessively
                 Key: REEF-1837
                 URL: https://issues.apache.org/jira/browse/REEF-1837
             Project: REEF
          Issue Type: Bug
          Components: REEF.NET Driver
            Reporter: Markus Weimer
            Priority: Minor


In {{IsParentAliveCancellationDetector}}, we generate a log line each time an upstream HTTP
server cannot be found, e.g.:

{code}
INFO: Using Yarn Configuration to retrieve RM address.
Microsoft.MachineLearning.Distributed.Core.Common.IsParentAliveCancellationDetector Error:
0 : 2017-07-23T18:00:28.6024001-07:00 0004
ERROR: Failed to initialize yarn configuration
Encountered error [System.ApplicationException: Unable to find RM Webapp Address from yarn-site.xml.
   at Org.Apache.REEF.Utilities.Runtime.Yarn.YarnConfiguration.GetYarnRMWebappEndpoints()
   at Microsoft.MachineLearning.Distributed.Core.Common.IsParentAliveCancellationDetector.InitYarnConfiguration(String
rmAddresses)]
{code}

This leads to excessively bloated logs. We should reduce the amount of logging here, e.g.
by assuming that the state of the YARN deployment doesn't change during execution of one job.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message