hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (HDDS-1499) OzoneManager Cache
Date Fri, 17 May 2019 22:06:00 GMT

     [ https://issues.apache.org/jira/browse/HDDS-1499?focusedWorklogId=244355&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-244355

ASF GitHub Bot logged work on HDDS-1499:

                Author: ASF GitHub Bot
            Created on: 17/May/19 22:05
            Start Date: 17/May/19 22:05
    Worklog Time Spent: 10m 
      Work Description: arp7 commented on pull request #798: HDDS-1499. OzoneManager Cache.
URL: https://github.com/apache/hadoop/pull/798#discussion_r285301711

 File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/TypedTable.java
 @@ -69,8 +79,34 @@ public boolean isEmpty() throws IOException {
     return rawTable.isEmpty();
+  /**
+   * Returns the value mapped to the given key in byte array or returns null
+   * if the key is not found.
+   *
+   * Caller's of this method should use synchronization mechanism, when
+   * accessing. First it will check from cache, if it has entry return the
+   * value, otherwise get from the RocksDB table.
+   *
+   * @param key metadata key
+   * @return VALUE
+   * @throws IOException
+   */
   public VALUE get(KEY key) throws IOException {
+    // Here the metadata lock will guarantee that cache is not updated for same
+    // key during get key.
+    CacheValue< VALUE > cacheValue = cache.get(new CacheKey<>(key));
+    if (cacheValue == null) {
+      // If no cache for the table or if it does not exist in cache get from
+      // RocksDB table.
+      return getFromTable(key);
+    } else {
+      // We have a value in cache, return the value.
+      return cacheValue.getValue();
 Review comment:
   cacheValue.getValue() can be a null to indicate deletion, correct?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

Issue Time Tracking

    Worklog Id:     (was: 244355)
    Time Spent: 9h 20m  (was: 9h 10m)

> OzoneManager Cache
> ------------------
>                 Key: HDDS-1499
>                 URL: https://issues.apache.org/jira/browse/HDDS-1499
>             Project: Hadoop Distributed Data Store
>          Issue Type: Sub-task
>          Components: Ozone Manager
>            Reporter: Bharat Viswanadham
>            Assignee: Bharat Viswanadham
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 9h 20m
>  Remaining Estimate: 0h
> In this Jira, we shall implement a cache for Table.
> As with OM HA, we are planning to implement double buffer implementation to flush transaction
in a batch, instead of using rocksdb put() for every operation. When this comes in to place
we need cache in OzoneManager HA to handle/server the requests for validation/returning responses.
> This Jira will implement Cache as an integral part of the table. In this way users using
this table does not need to handle like check cache/db. For this, we can update get API in
the table to handle the cache.
> This Jira will implement:
>  # Cache as a part of each Table.
>  # Uses this cache in get().
>  # Exposes api for cleanup, add entries to cache.
> Usage to add the entries in to cache will be done in further jira's.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message