From issues-return-147431-archive-asf-public=cust-asf.ponee.io@flink.apache.org Wed Jan 10 19:14:07 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id B75A118072F for ; Wed, 10 Jan 2018 19:14:07 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id A6C7A160C2E; Wed, 10 Jan 2018 18:14:07 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EBF33160C1E for ; Wed, 10 Jan 2018 19:14:06 +0100 (CET) Received: (qmail 1002 invoked by uid 500); 10 Jan 2018 18:14:06 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 993 invoked by uid 99); 10 Jan 2018 18:14:06 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Jan 2018 18:14:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 849031A03E1 for ; Wed, 10 Jan 2018 18:14:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.911 X-Spam-Level: X-Spam-Status: No, score=-99.911 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id gAIcZgw3ZKJW for ; Wed, 10 Jan 2018 18:14:04 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 992735FB93 for ; Wed, 10 Jan 2018 18:14:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id B5002E00A7 for ; Wed, 10 Jan 2018 18:14:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 6F0FF274D3 for ; Wed, 10 Jan 2018 18:14:00 +0000 (UTC) Date: Wed, 10 Jan 2018 18:14:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@flink.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (FLINK-5823) Store Checkpoint Root Metadata in StateBackend (not in HA custom store) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/FLINK-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16320756#comment-16320756 ] ASF GitHub Bot commented on FLINK-5823: --------------------------------------- Github user StephanEwen commented on a diff in the pull request: https://github.com/apache/flink/pull/5248#discussion_r160756860 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/CheckpointStorage.java --- @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.state; + +import javax.annotation.Nullable; + +import java.io.IOException; + +/** + * CheckpointStorage implements the durable storage of checkpoint data and metadata streams. + * An individual checkpoint or savepoint is stored to a {@link CheckpointStorageLocation}, + * created by this class. + */ +public interface CheckpointStorage { + + /** + * Checks whether this backend supports highly available storage of data. + * + *

Some state backends may offer support for that with default settings, which makes them + * suitable for zero-config prototyping, but not for actual production setups. + */ + boolean supportsHighlyAvailableStorage(); + + /** + * Checks whether the storage has a default savepoint location configured. + */ + boolean hasDefaultSavepointLocation(); + + /** + * Resolves the given pointer to a checkpoint/savepoint into a state handle from which the + * checkpoint metadata can be read. If the state backend cannot understand the format of + * the pointer (for example because it was created by a different state backend) this method + * should throw an {@code IOException}. + * + * @param pointer The pointer to resolve. + * @return The state handler from which one can read the checkpoint metadata. + * + * @throws IOException Thrown, if the state backend does not understand the pointer, or if + * the pointer could not be resolved due to an I/O error. + */ + StreamStateHandle resolveCheckpoint(String pointer) throws IOException; --- End diff -- I was wondering about that as well. My thinking was that in the end, what we get is just Strings (from the command line, from ZooKeeper, from the REST interface) and there is no point before that method that could parse/convert the string into something state backend specific. So we have to deal with Strings anyways. Wrapping them in another class felt not too useful to me here. > Store Checkpoint Root Metadata in StateBackend (not in HA custom store) > ----------------------------------------------------------------------- > > Key: FLINK-5823 > URL: https://issues.apache.org/jira/browse/FLINK-5823 > Project: Flink > Issue Type: Sub-task > Components: State Backends, Checkpointing > Reporter: Stephan Ewen > Assignee: Stephan Ewen > Priority: Blocker > Fix For: 1.5.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)