zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mina...@apache.org
Subject zeppelin git commit: [ZEPPELIN-1961] Improve stability of sync when get fails
Date Mon, 16 Jan 2017 02:17:46 GMT
Repository: zeppelin
Updated Branches:
  refs/heads/branch-0.7 a3d731619 -> 20c3cbccf


[ZEPPELIN-1961] Improve stability of sync when get fails

### What is this PR for?
This is to improve the stability of sync mechanism when `get` from some backend storage fails
(e.g. corrupt file, network issues).

### What type of PR is it?
Bug Fix |  Hot Fix

### Todos
* [x] - handle exception

### What is the Jira issue?
[ZEPPELIN-1961](https://issues.apache.org/jira/browse/ZEPPELIN-1961)

### How should this be tested?
CI green

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Khalid Huseynov <khalidhnv@gmail.com>

Closes #1895 from khalidhuseynov/fix-stability/sync-fail and squashes the following commits:

aa1e199 [Khalid Huseynov] catch failed get command

(cherry picked from commit e94d5c0fb66ee0954a74450002c5b81b8a7331a7)
Signed-off-by: Mina Lee <minalee@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/20c3cbcc
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/20c3cbcc
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/20c3cbcc

Branch: refs/heads/branch-0.7
Commit: 20c3cbccfd0ac4e58637bd4324774b7e3ab536c9
Parents: a3d7316
Author: Khalid Huseynov <khalidhnv@gmail.com>
Authored: Thu Jan 12 15:12:29 2017 -0800
Committer: Mina Lee <minalee@apache.org>
Committed: Mon Jan 16 11:17:39 2017 +0900

----------------------------------------------------------------------
 .../zeppelin/notebook/repo/NotebookRepoSync.java      | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/20c3cbcc/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSync.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSync.java
b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSync.java
index 73b25e9..8553349 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSync.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSync.java
@@ -330,8 +330,7 @@ public class NotebookRepoSync implements NotebookRepo {
 
   private Map<String, List<String>> notesCheckDiff(List<NoteInfo> sourceNotes,
       NotebookRepo sourceRepo, List<NoteInfo> destNotes, NotebookRepo destRepo,
-      AuthenticationInfo subject)
-      throws IOException {
+      AuthenticationInfo subject) {
     List <String> pushIDs = new ArrayList<>();
     List <String> pullIDs = new ArrayList<>();
     List <String> delDstIDs = new ArrayList<>();
@@ -341,9 +340,14 @@ public class NotebookRepoSync implements NotebookRepo {
     for (NoteInfo snote : sourceNotes) {
       dnote = containsID(destNotes, snote.getId());
       if (dnote != null) {
-        /* note exists in source and destination storage systems */
-        sdate = lastModificationDate(sourceRepo.get(snote.getId(), subject));
-        ddate = lastModificationDate(destRepo.get(dnote.getId(), subject));
+        try {
+          /* note exists in source and destination storage systems */
+          sdate = lastModificationDate(sourceRepo.get(snote.getId(), subject));
+          ddate = lastModificationDate(destRepo.get(dnote.getId(), subject));
+        } catch (IOException e) {
+          LOG.error("Cannot access previously listed note {} from storage ", dnote.getId(),
e);
+          continue;
+        }
 
         if (sdate.compareTo(ddate) != 0) {
           if (sdate.after(ddate) || oneWaySync) {


Mime
View raw message