Return-Path: X-Original-To: apmail-flink-commits-archive@minotaur.apache.org Delivered-To: apmail-flink-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7145C185B9 for ; Mon, 6 Jul 2015 14:02:07 +0000 (UTC) Received: (qmail 44741 invoked by uid 500); 6 Jul 2015 14:02:07 -0000 Delivered-To: apmail-flink-commits-archive@flink.apache.org Received: (qmail 44712 invoked by uid 500); 6 Jul 2015 14:02:07 -0000 Mailing-List: contact commits-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list commits@flink.apache.org Received: (qmail 44703 invoked by uid 99); 6 Jul 2015 14:02:07 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Jul 2015 14:02:07 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 41729E0286; Mon, 6 Jul 2015 14:02:07 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: sewen@apache.org To: commits@flink.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: flink-web git commit: Update FAQ entry on Fault Tolerance Date: Mon, 6 Jul 2015 14:02:07 +0000 (UTC) Repository: flink-web Updated Branches: refs/heads/asf-site 5dcbfe106 -> 9760e5eb5 Update FAQ entry on Fault Tolerance Fix various links Project: http://git-wip-us.apache.org/repos/asf/flink-web/repo Commit: http://git-wip-us.apache.org/repos/asf/flink-web/commit/9760e5eb Tree: http://git-wip-us.apache.org/repos/asf/flink-web/tree/9760e5eb Diff: http://git-wip-us.apache.org/repos/asf/flink-web/diff/9760e5eb Branch: refs/heads/asf-site Commit: 9760e5eb5f3159657f2f9a80fdc5c597564f8c80 Parents: 5dcbfe1 Author: Stephan Ewen Authored: Mon Jul 6 16:01:27 2015 +0200 Committer: Stephan Ewen Committed: Mon Jul 6 16:01:27 2015 +0200 ---------------------------------------------------------------------- content/community.html | 4 ++-- content/faq.html | 10 ++++++---- faq.md | 11 +++++++---- 3 files changed, 15 insertions(+), 10 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flink-web/blob/9760e5eb/content/community.html ---------------------------------------------------------------------- diff --git a/content/community.html b/content/community.html index ec76633..ce85100 100644 --- a/content/community.html +++ b/content/community.html @@ -343,7 +343,7 @@ aljoscha - + Andra Lungu Committer andra @@ -361,7 +361,7 @@ mxm - + Chiwan Park Committer chiwanpark http://git-wip-us.apache.org/repos/asf/flink-web/blob/9760e5eb/content/faq.html ---------------------------------------------------------------------- diff --git a/content/faq.html b/content/faq.html index b1e7c3d..78dd9b1 100644 --- a/content/faq.html +++ b/content/faq.html @@ -581,15 +581,17 @@ where it stores the Flink jar and configuration file.

-

Flink can restart failed jobs. Mid-query fault tolerance will go into the -open source project in the next versions.

+

For streaming programs, Flink has a novel approach to draw periodic snapshots of the streaming dataflow state and use those for recovery. +This mechanism is both efficient and flexible. See the documentation on streaming fault tolerance for details.

+ +

For batch processing programs, Flink remembers the programs sequence of transformations and can restart failed jobs.

Are Hadoop-like utilities, such as Counters and the DistributedCache supported?

Flink’s Accumulators work very similar like -[Hadoop’s counters, but are more powerful.

+Hadoop’s counters, but are more powerful.

-

Flink has a https://github.com/apache/flink/tree/master//flink-core/src/main/java/org/apache/flink/api/common/cache/DistributedCache.java that is deeply integrated with the APIs. Please refer to the https://github.com/apache/flink/tree/master//flink-java/src/main/java/org/apache/flink/api/java/ExecutionEnvironment.java#L561 for details on how to use it.

+

Flink has a Distributed Cache that is deeply integrated with the APIs. Please refer to the JavaDocs for details on how to use it.

In order to make data sets available on all tasks, we encourage you to use Broadcast Variables instead. They are more efficient and easier to use than the distributed cache.

http://git-wip-us.apache.org/repos/asf/flink-web/blob/9760e5eb/faq.md ---------------------------------------------------------------------- diff --git a/faq.md b/faq.md index 85299e8..5e44742 100644 --- a/faq.md +++ b/faq.md @@ -384,14 +384,17 @@ where it stores the Flink jar and configuration file. ### What kind of fault-tolerance does Flink provide? -Flink can restart failed jobs. Mid-query fault tolerance will go into the -open source project in the next versions. +For streaming programs, Flink has a novel approach to draw periodic snapshots of the streaming dataflow state and use those for recovery. +This mechanism is both efficient and flexible. See the documentation on [streaming fault tolerance]({{ site.docs-snapshot }}/internals/stream_checkpointing.html) for details. + +For batch processing programs, Flink remembers the programs sequence of transformations and can restart failed jobs. + ### Are Hadoop-like utilities, such as Counters and the DistributedCache supported? [Flink's Accumulators]({{ site.docs-snapshot }}/apis/programming_guide.html#accumulators--counters) work very similar like -[Hadoop's counters, but are more powerful. +Hadoop's counters, but are more powerful. -Flink has a {% github /flink-core/src/main/java/org/apache/flink/api/common/cache/DistributedCache.java "Distributed Cache" %} that is deeply integrated with the APIs. Please refer to the {% github /flink-java/src/main/java/org/apache/flink/api/java/ExecutionEnvironment.java#L561 "JavaDocs" %} for details on how to use it. +Flink has a [Distributed Cache](https://github.com/apache/flink/tree/master/flink-core/src/main/java/org/apache/flink/api/common/cache/DistributedCache.java) that is deeply integrated with the APIs. Please refer to the [JavaDocs](https://github.com/apache/flink/tree/master/flink-java/src/main/java/org/apache/flink/api/java/ExecutionEnvironment.java#L831) for details on how to use it. In order to make data sets available on all tasks, we encourage you to use [Broadcast Variables]({{ site.docs-snapshot }}/apis/programming_guide.html#broadcast-variables) instead. They are more efficient and easier to use than the distributed cache.