ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sid Wagle" <swa...@hortonworks.com>
Subject Re: Review Request 40929: AMBARI-14140 : Test and Adopt FIFO compaction policy for AMS high load tables
Date Thu, 03 Dec 2015 22:23:58 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40929/#review108895
-----------------------------------------------------------



ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector (line 41)
<https://reviews.apache.org/r/40929/#comment168359>

    PID dir paths can be changed.



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
(line 520)
<https://reviews.apache.org/r/40929/#comment168358>

    Naming convention looks a little inconsistent.
    How about timeline.metrics.hbase.fifo.compaction.enabled


- Sid Wagle


On Dec. 3, 2015, 9:59 p.m., Aravindan Vijayan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40929/
> -----------------------------------------------------------
> 
> (Updated Dec. 3, 2015, 9:59 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-14140
>     https://issues.apache.org/jira/browse/AMBARI-14140
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> The following patch adopts the FIFO compaction policy for the AMS high load tables -
METRIC_RECORD, METRIC_RECORD_MINUTE, METRIC_AGGREGATE.
> 
> Rationale for adopting FIFO policy
> FIFO compaction policy selects only files which have all cells expired. The column family
MUST have non-default TTL.
> Since currently AMS does not do any real compaction, it does not use CPU and IO (disk
and network), it does not evict hot data from a block cache. The expected result is improved
throughput and latency both write and read.
> 
> Work done
> Added UI configs and methods to enable and disable compaction policy. The new UI config
is hbase.fifo.compaction.policy.enabled, which can take true/false values. The enabling/disabling
is done through HBase shell command line. 
> 
> If hbase.fifo.compaction.policy.enabled = true
>   If 'fifo_enabled' stub file is not present
>      enable_fifo_compaction + create stub file
> 
> If hbase.fifo.compaction.policy.enabled = false
>   If fifo_enabled' stub file is present
>      disable_fifo_compaction + remove stub file
> 
> The above check has been added to the HBase normalization enabling mechanism as well.
> 
> 
> Diffs
> -----
> 
>   ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector 52fe9ea

>   ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog213.java
b614b02 
>   ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
5abde3f 
>   ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
04b7806 
>   ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
302b376 
>   ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog213Test.java
ee01ab0 
> 
> Diff: https://reviews.apache.org/r/40929/diff/
> 
> 
> Testing
> -------
> 
> Manually tested different workflows from the UI.
> 
> Submitted patch through Apache.
> 
> Relevant unit tests pass. Other unit tests currently running.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message