hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (HDDS-1827) Load Snapshot info when OM Ratis server starts
Date Fri, 16 Aug 2019 17:52:00 GMT

     [ https://issues.apache.org/jira/browse/HDDS-1827?focusedWorklogId=296469&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-296469
]

ASF GitHub Bot logged work on HDDS-1827:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 16/Aug/19 17:51
            Start Date: 16/Aug/19 17:51
    Worklog Time Spent: 10m 
      Work Description: arp7 commented on pull request #1130: HDDS-1827. Load Snapshot info
when OM Ratis server starts.
URL: https://github.com/apache/hadoop/pull/1130#discussion_r314825528
 
 

 ##########
 File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/OMRatisSnapshotInfo.java
 ##########
 @@ -0,0 +1,133 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership.  The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.hadoop.ozone.om.ratis;
+
+import org.apache.ratis.server.protocol.TermIndex;
+import org.apache.ratis.server.storage.FileInfo;
+import org.apache.ratis.statemachine.SnapshotInfo;
+import org.apache.hadoop.ozone.om.OzoneManager;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.util.List;
+
+import static org.apache.hadoop.ozone.OzoneConsts.OM_RATIS_SNAPSHOT_INDEX;
+
+/**
+ * This class captures the {@link OzoneManager#snapshotIndex} and term of the
+ * snapshot.
+ * Ratis server loads the snapshotInfo during startup and updates the
+ * lastApplied index to this snapshotIndex. OM SnaphsotInfo does not contain
+ * any files. It is used only to store/ update the last applied index and term.
+ */
+public class OMRatisSnapshotInfo implements SnapshotInfo {
+
+  static final Logger LOG = LoggerFactory.getLogger(OMRatisSnapshotInfo.class);
+
+  private volatile long term = 0;
+  private volatile long snapshotIndex = -1;
+
+  private final File ratisSnapshotFile;
+
+  private final static String TERM_PREFIX = "term";
 
 Review comment:
   It may be better to serialize using YAML. We can commit this for now but let's file a follow
up jira. 
   
   It will likely be easier to extend in the future.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 296469)
    Time Spent: 1.5h  (was: 1h 20m)

> Load Snapshot info when OM Ratis server starts
> ----------------------------------------------
>
>                 Key: HDDS-1827
>                 URL: https://issues.apache.org/jira/browse/HDDS-1827
>             Project: Hadoop Distributed Data Store
>          Issue Type: Sub-task
>            Reporter: Hanisha Koneru
>            Assignee: Hanisha Koneru
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> When Ratis server is starting it looks for the latest snapshot to load it. Even though
OM does not save snapshots via Ratis, we need to load the saved snaphsot index into Ratis
so that the LogAppender knows to not look for logs before the snapshot index. Otherwise, Ratis
will replay the logs from beginning every time it starts up.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message