Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Fri, 12 May 2017 11:45:04 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.13062769.1491797008000.196780.1494589504730@Atlassian.JIRA>
In-Reply-To: <JIRA.13062769.1491797008000@Atlassian.JIRA>
References: <JIRA.13062769.1491797008000@Atlassian.JIRA> <JIRA.13062769.1491797008714@jira-lw-us.apache.org>
Subject: [jira] [Commented] (FLINK-6284) Incorrect sorting of completed
 checkpoints in ZooKeeperCompletedCheckpointStore
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 12 May 2017 11:45:12 -0000


    [ https://issues.apache.org/jira/browse/FLINK-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008000#comment-16008000 ] 

ASF GitHub Bot commented on FLINK-6284:
---------------------------------------

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3881#discussion_r116208844
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/zookeeper/ZooKeeperStateHandleStore.java ---
    @@ -346,17 +346,20 @@ public int exists(String pathInZooKeeper) throws Exception {
     			} else {
     				// Initial cVersion (number of changes to the children of this node)
     				int initialCVersion = stat.getCversion();
    -
    -				List<String> children = ZKPaths.getSortedChildren(
    -						client.getZookeeperClient().getZooKeeper(),
    -						ZKPaths.fixForNamespace(client.getNamespace(), "/"));
    -
    -				for (String path : children) {
    -					path = "/" + path;
    +				List<String> childrenInStr =
    +					client.getZookeeperClient().getZooKeeper().
    +						getChildren(ZKPaths.fixForNamespace(client.getNamespace(), "/"), false);
    +				List<Long> children = new ArrayList<Long>(childrenInStr.size());
    +				for(String childNode : childrenInStr) {
    +					children.add(new Long(childNode));
    --- End diff --
    
    I'm not sure whether we can assume that the children paths are always longs. In the general case this is not true (see `ZooKeeperMesosWorkerStore`).


> Incorrect sorting of completed checkpoints in ZooKeeperCompletedCheckpointStore
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-6284
>                 URL: https://issues.apache.org/jira/browse/FLINK-6284
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>            Reporter: Xiaogang Shi
>            Priority: Blocker
>             Fix For: 1.3.0
>
>
> Now all completed checkpoints are sorted in their paths when they are recovered in {{ZooKeeperCompletedCheckpointStore}} . In the cases where the latest checkpoint's id is not the largest in lexical order (e.g., "100" is smaller than "99" in lexical order), Flink will not recover from the latest completed checkpoint.
> The problem can be easily observed by setting the checkpoint ids in {{ZooKeeperCompletedCheckpointStoreITCase#testRecover()}} to be 99, 100 and 101. 
> To fix the problem, we should explicitly sort found checkpoints in their checkpoint ids, without the usage of {{ZooKeeperStateHandleStore#getAllSortedByName()}}


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)