ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cabir Zounaidou" <czounai...@hortonworks.com>
Subject Re: Review Request 27802: Rest api's for host component logs.
Date Thu, 13 Nov 2014 01:01:40 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27802/
-----------------------------------------------------------

(Updated Nov. 13, 2014, 1:01 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, John Speidel, Mahadev Konar,
Nate Cole, Sid Wagle, and Yusaku Sako.


Changes
-------

Applied most of the changes suggested in the review.


Bugs: AMBARI-4083
    https://issues.apache.org/jira/browse/AMBARI-4083


Repository: ambari


Description (updated)
-------

The host component log implements the following two rest api's.

1. To retrieve the log entries. 
  /clusters/{cluster}/hosts/{hostname}/host_components_logs/{component}
  This api will retrieve the logs if available for the host component from HDFS.  It can fetch
only maximum of 5120 bytes.  The window can be adjusted using the query parameters 'offset'
and 'length'.  It also provides simple filtering using 'level' query parameter.
  Sample response will look like below:
  {
  "href" : "http://cabir-logs-1:8080/api/v1/clusters/cl1/hosts/cabir-logs-1.c.pramod-thangali.internal/host_components_logs/HBASE_MASTER",
  "length" : 5120,
  "level" : null,
  "offset" : 125952,
  "size" : 131072,
  "logs" : [
    {
      "timestamp" : "2014-11-12 20:24:58,508",
      "level" : "DEBUG",
      "thread" : "cabir-logs-1.c.pramod-thangali.internal,60000,1415771549794-BalancerChore",
      "message" : "balancer.BaseLoadBalancer: Not running balancer because only 1 active regionserver(s)"
    },
    {
      "timestamp" : "2014-11-12 20:29:58,509",
      "level" : "DEBUG",
      "thread" : "cabir-logs-1.c.pramod-thangali.internal,60000,1415771549794-BalancerChore",
      "message" : "balancer.BaseLoadBalancer: Not running balancer because only 1 active regionserver(s)"
    },
    {
      "timestamp" : "2014-11-12 20:34:58,516",
      "level" : "DEBUG",
      "thread" : "cabir-logs-1.c.pramod-thangali.internal,60000,1415771549794-BalancerChore",
      "message" : "balancer.BaseLoadBalancer: Not running balancer because only 1 active regionserver(s)"
    },
    {
      "timestamp" : "2014-11-12 20:39:58,509",
      "level" : "DEBUG",
      "thread" : "cabir-logs-1.c.pramod-thangali.internal,60000,1415771549794-BalancerChore",
      "message" : "balancer.BaseLoadBalancer: Not running balancer because only 1 active regionserver(s)"
    }
    :
  ]
}
2. To download the log file from HDFS use the following the api
  /clusters/{cluster}/hosts/{hostname}/host_components_logs/{component}?format=file
  The response will automatically download the file with the HDFS file name.


Diffs (updated)
-----

  ambari-agent/src/main/python/ambari_agent/Controller.py dc3a1cf 
  ambari-server/pom.xml 4e7477e 
  ambari-server/src/main/java/org/apache/ambari/server/agent/AgentCommand.java e2f013d 
  ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 82b642b

  ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatResponse.java 56b4f18

  ambari-server/src/main/java/org/apache/ambari/server/agent/LogConfigCommand.java PRE-CREATION

  ambari-server/src/main/java/org/apache/ambari/server/agent/RegistrationResponse.java 8a24560

  ambari-server/src/main/java/org/apache/ambari/server/api/query/render/BinaryRenderer.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentLogResourceDefinition.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java
6dc9e2d 
  ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
9ad37ec 
  ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 2a137c9

  ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d

  ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java
4990ad7 
  ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java c51722c

  ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/OctetStreamSerializer.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java 9970d9a

  ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d74510a

  ambari-server/src/main/java/org/apache/ambari/server/controller/HostComponentLogResponse.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
ae20f56 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProvider.java
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java dbac906

  ambari-server/src/main/java/org/apache/ambari/server/state/LogConfig.java PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/state/LogConfigHandler.java PRE-CREATION

  ambari-server/src/main/java/org/apache/ambari/server/state/LogDefinition.java PRE-CREATION

  ambari-server/src/main/java/org/apache/ambari/server/state/LogEntry.java PRE-CREATION 
  ambari-server/src/main/resources/key_properties.json c1a6636 
  ambari-server/src/main/resources/log_handler_config.json PRE-CREATION 
  ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
6ab35c2 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostComponentLogResourceProviderTest.java
PRE-CREATION 

Diff: https://reviews.apache.org/r/27802/diff/


Testing
-------

1. Ran the unit tests.
2. Manually ran the retrieve host component logs without any query parameters.
3. Manually ran the retrieve host component logs with offset and length.
4. Manually ran the retrieve host component logs with level filter.
5. Manually ran the retrieve host component logs with webHDFS disabled.
6. Manually ran the download host component log file with webHDFS enabled.
7. Manually ran the download host component log file with webHDFS disabled.


Thanks,

Cabir Zounaidou


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message