Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 35D48200C0F for ; Wed, 18 Jan 2017 15:03:20 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 346A9160B34; Wed, 18 Jan 2017 14:03:20 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id AF74C160B59 for ; Wed, 18 Jan 2017 15:03:18 +0100 (CET) Received: (qmail 83671 invoked by uid 500); 18 Jan 2017 14:03:17 -0000 Mailing-List: contact commits-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list commits@flink.apache.org Received: (qmail 74115 invoked by uid 99); 18 Jan 2017 14:00:13 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Jan 2017 14:00:13 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id C5D0DF404C; Wed, 18 Jan 2017 14:00:03 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: uce@apache.org To: commits@flink.apache.org Date: Wed, 18 Jan 2017 14:00:54 -0000 Message-Id: <085ac663808747beae96544e27bbb741@git.apache.org> In-Reply-To: <7f404cb01cdd4af5ac1587c8a9908179@git.apache.org> References: <7f404cb01cdd4af5ac1587c8a9908179@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [54/84] [abbrv] flink-web git commit: Rebuild website archived-at: Wed, 18 Jan 2017 14:03:20 -0000 http://git-wip-us.apache.org/repos/asf/flink-web/blob/61adc137/content/q/quickstart-SNAPSHOT.sh ---------------------------------------------------------------------- diff --git a/content/q/quickstart-SNAPSHOT.sh b/content/q/quickstart-SNAPSHOT.sh new file mode 100755 index 0000000..1b1f2d2 --- /dev/null +++ b/content/q/quickstart-SNAPSHOT.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash + +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +PACKAGE=quickstart + +mvn archetype:generate \ + -DarchetypeGroupId=org.apache.flink \ + -DarchetypeArtifactId=flink-quickstart-java \ + -DarchetypeVersion=1.2-SNAPSHOT \ + -DgroupId=org.myorg.quickstart \ + -DartifactId=$PACKAGE \ + -Dversion=0.1 \ + -Dpackage=org.myorg.quickstart \ + -DinteractiveMode=false \ + -DarchetypeCatalog=https://repository.apache.org/content/repositories/snapshots/ + +# +# Give some guidance +# +echo -e "\\n\\n" +echo -e "\\tA sample quickstart Flink Job has been created." +echo -e "\\tSwitch into the directory using" +echo -e "\\t\\t cd $PACKAGE" +echo -e "\\tImport the project there using your favorite IDE (Import it as a maven project)" +echo -e "\\tBuild a jar inside the directory using" +echo -e "\\t\\t mvn clean package" +echo -e "\\tYou will find the runnable jar in $PACKAGE/target" +echo -e "\\tConsult our website if you have any troubles: http://flink.apache.org/community.html#mailing-lists" +echo -e "\\n\\n" http://git-wip-us.apache.org/repos/asf/flink-web/blob/61adc137/content/q/quickstart-scala-SNAPSHOT.sh ---------------------------------------------------------------------- diff --git a/content/q/quickstart-scala-SNAPSHOT.sh b/content/q/quickstart-scala-SNAPSHOT.sh new file mode 100755 index 0000000..9cfeb2e --- /dev/null +++ b/content/q/quickstart-scala-SNAPSHOT.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash + +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +PACKAGE=quickstart + +mvn archetype:generate \ + -DarchetypeGroupId=org.apache.flink \ + -DarchetypeArtifactId=flink-quickstart-scala \ + -DarchetypeVersion=1.2-SNAPSHOT \ + -DgroupId=org.myorg.quickstart \ + -DartifactId=$PACKAGE \ + -Dversion=0.1 \ + -Dpackage=org.myorg.quickstart \ + -DinteractiveMode=false \ + -DarchetypeCatalog=https://repository.apache.org/content/repositories/snapshots/ + +# +# Give some guidance +# +echo -e "\\n\\n" +echo -e "\\tA sample quickstart Flink Job has been created." +echo -e "\\tSwitch into the directory using" +echo -e "\\t\\t cd $PACKAGE" +echo -e "\\tImport the project there using your favorite IDE (Import it as a maven project)" +echo -e "\\tBuild a jar inside the directory using" +echo -e "\\t\\t mvn clean package" +echo -e "\\tYou will find the runnable jar in $PACKAGE/target" +echo -e "\\tConsult our website if you have any troubles: http://flink.apache.org/community.html#mailing-lists" +echo -e "\\n\\n" http://git-wip-us.apache.org/repos/asf/flink-web/blob/61adc137/content/q/quickstart-scala.sh ---------------------------------------------------------------------- diff --git a/content/q/quickstart-scala.sh b/content/q/quickstart-scala.sh new file mode 100755 index 0000000..7dcbe34 --- /dev/null +++ b/content/q/quickstart-scala.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash + +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +PACKAGE=quickstart + +mvn archetype:generate \ + -DarchetypeGroupId=org.apache.flink \ + -DarchetypeArtifactId=flink-quickstart-scala \ + -DarchetypeVersion=1.1.4 \ + -DgroupId=org.myorg.quickstart \ + -DartifactId=$PACKAGE \ + -Dversion=0.1 \ + -Dpackage=org.myorg.quickstart \ + -DinteractiveMode=false + +# +# Give some guidance +# +echo -e "\\n\\n" +echo -e "\\tA sample quickstart Flink Job has been created." +echo -e "\\tSwitch into the directory using" +echo -e "\\t\\t cd $PACKAGE" +echo -e "\\tImport the project there using your favorite IDE (Import it as a maven project)" +echo -e "\\tBuild a jar inside the directory using" +echo -e "\\t\\t mvn clean package" +echo -e "\\tYou will find the runnable jar in $PACKAGE/target" +echo -e "\\tConsult our website if you have any troubles: http://flink.apache.org/community.html#mailing-lists" +echo -e "\\n\\n" http://git-wip-us.apache.org/repos/asf/flink-web/blob/61adc137/content/q/quickstart.sh ---------------------------------------------------------------------- diff --git a/content/q/quickstart.sh b/content/q/quickstart.sh new file mode 100755 index 0000000..972400a --- /dev/null +++ b/content/q/quickstart.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash + +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +PACKAGE=quickstart + +mvn archetype:generate \ + -DarchetypeGroupId=org.apache.flink \ + -DarchetypeArtifactId=flink-quickstart-java \ + -DarchetypeVersion=1.1.3 \ + -DgroupId=org.myorg.quickstart \ + -DartifactId=$PACKAGE \ + -Dversion=0.1 \ + -Dpackage=org.myorg.quickstart \ + -DinteractiveMode=false + +# +# Give some guidance +# +echo -e "\\n\\n" +echo -e "\\tA sample quickstart Flink Job has been created." +echo -e "\\tSwitch into the directory using" +echo -e "\\t\\t cd $PACKAGE" +echo -e "\\tImport the project there using your favorite IDE (Import it as a maven project)" +echo -e "\\tBuild a jar inside the directory using" +echo -e "\\t\\t mvn clean package" +echo -e "\\tYou will find the runnable jar in $PACKAGE/target" +echo -e "\\tConsult our website if you have any troubles: http://flink.apache.org/community.html#mailing-lists" +echo -e "\\n\\n" http://git-wip-us.apache.org/repos/asf/flink-web/blob/61adc137/content/q/sbt-quickstart.sh ---------------------------------------------------------------------- diff --git a/content/q/sbt-quickstart.sh b/content/q/sbt-quickstart.sh new file mode 100755 index 0000000..78abcb8 --- /dev/null +++ b/content/q/sbt-quickstart.sh @@ -0,0 +1,346 @@ +#!/bin/bash + +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +declare -r TRUE=0 +declare -r FALSE=1 + +# takes a string and returns true if it seems to represent "yes" +function isYes() { + local x=$1 + [ $x = "y" -o $x = "Y" -o $x = "yes" ] && echo $TRUE; return + echo $FALSE +} + +function mkDir() { + local x=$1 + echo ${x// /-} | tr '[:upper:]' '[:lower:]' +} + +function mkPackage() { + local x=$1 + echo ${x//./\/} +} + +defaultProjectName="Flink Project" +defaultOrganization="org.example" +defaultVersion="0.1-SNAPSHOT" +defaultScalaVersion="2.11.7" +defaultFlinkVersion="1.1.3" + +echo "This script creates a Flink project using Scala and SBT." + +while [ $TRUE ]; do + + echo "" + read -p "Project name ($defaultProjectName): " projectName + projectName=${projectName:-$defaultProjectName} + read -p "Organization ($defaultOrganization): " organization + organization=${organization:-$defaultOrganization} + read -p "Version ($defaultVersion): " version + version=${version:-$defaultVersion} + read -p "Scala version ($defaultScalaVersion): " scalaVersion + scalaVersion=${scalaVersion:-$defaultScalaVersion} + read -p "Flink version ($defaultFlinkVersion): " flinkVersion + flinkVersion=${flinkVersion:-$defaultFlinkVersion} + + echo "" + echo "-----------------------------------------------" + echo "Project Name: $projectName" + echo "Organization: $organization" + echo "Version: $version" + echo "Scala version: $scalaVersion" + echo "Flink version: $flinkVersion" + echo "-----------------------------------------------" + read -p "Create Project? (Y/n): " createProject + createProject=${createProject:-y} + + [ "$(isYes "$createProject")" = "$TRUE" ] && break + +done + +directoryName=$(mkDir "$projectName") + +echo "Creating Flink project under $directoryName" + +mkdir -p ${directoryName}/src/main/{resources,scala} +mkdir -p ${directoryName}/project + +# Create the README file + +echo "A Flink application project using Scala and SBT. + +To run and test your application use SBT invoke: 'sbt run' + +In order to run your application from within IntelliJ, you have to select the classpath of the 'mainRunner' module in the run/debug configurations. Simply open 'Run -> Edit configurations...' and then select 'mainRunner' from the "Use classpath of module" dropbox." > ${directoryName}/README + +# Create the build.sbt file + +echo "resolvers in ThisBuild ++= Seq(\"Apache Development Snapshot Repository\" at \"https://repository.apache.org/content/repositories/snapshots/\", Resolver.mavenLocal) + +name := \"$projectName\" + +version := \"$version\" + +organization := \"$organization\" + +scalaVersion in ThisBuild := \"$scalaVersion\" + +val flinkVersion = \"$flinkVersion\" + +val flinkDependencies = Seq( + \"org.apache.flink\" %% \"flink-scala\" % flinkVersion % \"provided\", + \"org.apache.flink\" %% \"flink-streaming-scala\" % flinkVersion % \"provided\") + +lazy val root = (project in file(\".\")). + settings( + libraryDependencies ++= flinkDependencies + ) + +mainClass in assembly := Some(\"$organization.Job\") + +// make run command include the provided dependencies +run in Compile <<= Defaults.runTask(fullClasspath in Compile, mainClass in (Compile, run), runner in (Compile, run)) + +// exclude Scala library from assembly +assemblyOption in assembly := (assemblyOption in assembly).value.copy(includeScala = false)" > ${directoryName}/build.sbt + +# Create idea.sbt file for mainRunner module for IntelliJ + +echo "lazy val mainRunner = project.in(file(\"mainRunner\")).dependsOn(RootProject(file(\".\"))).settings( + // we set all provided dependencies to none, so that they are included in the classpath of mainRunner + libraryDependencies := (libraryDependencies in RootProject(file(\".\"))).value.map{ + module => + if (module.configurations.equals(Some(\"provided\"))) { + module.copy(configurations = None) + } else { + module + } + } +)" > ${directoryName}/idea.sbt + +# Create assembly plugin file + +echo "addSbtPlugin(\"com.eed3si9n\" % \"sbt-assembly\" % \"0.14.1\")" > ${directoryName}/project/assembly.sbt + +# Create package structure + +mkdir -p ${directoryName}/src/main/scala/$(mkPackage $organization) + +# Create simple job skeleton + +echo "package $organization + +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * \"License\"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an \"AS IS\" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.flink.api.scala._ + +/** + * Skeleton for a Flink Job. + * + * You can also generate a .jar file that you can submit on your Flink + * cluster. Just type + * {{{ + * sbt clean assembly + * }}} + * in the projects root directory. You will find the jar in + * target/scala-2.11/Flink\ Project-assembly-0.1-SNAPSHOT.jar + * + */ +object Job { + def main(args: Array[String]) { + // set up the execution environment + val env = ExecutionEnvironment.getExecutionEnvironment + + /** + * Here, you can start creating your execution plan for Flink. + * + * Start with getting some data from the environment, like + * env.readTextFile(textPath); + * + * then, transform the resulting DataSet[String] using operations + * like: + * .filter() + * .flatMap() + * .join() + * .group() + * + * and many more. + * Have a look at the programming guide: + * + * http://flink.apache.org/docs/latest/programming_guide.html + * + * and the examples + * + * http://flink.apache.org/docs/latest/examples.html + * + */ + + + // execute program + env.execute(\"Flink Scala API Skeleton\") + } +}" > ${directoryName}/src/main/scala/$(mkPackage $organization)/Job.scala + +# Create WordCount example + +echo "package $organization + +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * \"License\"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an \"AS IS\" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.flink.api.scala._ + +/** + * Implements the \"WordCount\" program that computes a simple word occurrence histogram + * over some sample data + * + * This example shows how to: + * + * - write a simple Flink program. + * - use Tuple data types. + * - write and use user-defined functions. + */ +object WordCount { + def main(args: Array[String]) { + + // set up the execution environment + val env = ExecutionEnvironment.getExecutionEnvironment + + // get input data + val text = env.fromElements(\"To be, or not to be,--that is the question:--\", + \"Whether 'tis nobler in the mind to suffer\", \"The slings and arrows of outrageous fortune\", + \"Or to take arms against a sea of troubles,\") + + val counts = text.flatMap { _.toLowerCase.split(\"\\\\W+\") } + .map { (_, 1) } + .groupBy(0) + .sum(1) + + // execute and print result + counts.print() + + } +} +" > ${directoryName}/src/main/scala/$(mkPackage $organization)/WordCount.scala + +# Create SocketTextStreamWordCount example + +echo "package $organization + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * \"License\"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an \"AS IS\" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import org.apache.flink.streaming.api.scala._ + +/** + * This example shows an implementation of WordCount with data from a text socket. + * To run the example make sure that the service providing the text data is already up and running. + * + * To start an example socket text stream on your local machine run netcat from a command line, + * where the parameter specifies the port number: + * + * {{{ + * nc -lk 9999 + * }}} + * + * Usage: + * {{{ + * SocketTextStreamWordCount + * }}} + * + * This example shows how to: + * + * - use StreamExecutionEnvironment.socketTextStream + * - write a simple Flink Streaming program in scala. + * - write and use user-defined functions. + */ +object SocketTextStreamWordCount { + + def main(args: Array[String]) { + if (args.length != 2) { + System.err.println(\"USAGE:\nSocketTextStreamWordCount \") + return + } + + val hostName = args(0) + val port = args(1).toInt + + val env = StreamExecutionEnvironment.getExecutionEnvironment + + //Create streams for names and ages by mapping the inputs to the corresponding objects + val text = env.socketTextStream(hostName, port) + val counts = text.flatMap { _.toLowerCase.split(\"\\\\W+\") filter { _.nonEmpty } } + .map { (_, 1) } + .keyBy(0) + .sum(1) + + counts print + + env.execute(\"Scala SocketTextStreamWordCount Example\") + } + +} +" > ${directoryName}/src/main/scala/$(mkPackage $organization)/SocketTextStreamWordCount.scala http://git-wip-us.apache.org/repos/asf/flink-web/blob/61adc137/content/slides.html ---------------------------------------------------------------------- diff --git a/content/slides.html b/content/slides.html new file mode 100644 index 0000000..38e14d3 --- /dev/null +++ b/content/slides.html @@ -0,0 +1,265 @@ + + + + + + + + Apache Flink: Slides + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + +
+
+
+

Slides

+ +
+ + +
+ +

Training

+ +

Currently, there is a free training available by dataArtisans. Their training website contains slides and exercices with solutions. The slides are also available on SlideShare.

+ + + +

Flink Forward 2015 (October 12-13, 2015) was the first conference to bring together the Apache Flink developer and user community. You can find slides and videos of all talks on the Flink Forward 2015 page.

+ +

On September 12-14, 2016 the second edition of Flink Forward took place. All slides and videos are available on the Flink Forward 2016 page.

+ +

Slides

+ +

Note: Keep in mind that code examples on slides have a chance of being incomplete or outdated. Always refer to the latest documentation for an up to date reference.

+ +

2016

+ +
    +
  • Kostas Tzoumas & Stephan Ewen: Keynote -The maturing data streaming ecosystem and Apache Flink’s accelerated growth Flink Forward, Berlin September 2016: SlideShare
    • +
  • Robert Metzger: Connecting Apache Flink to the World - Reviewing the streaming connectors Flink Forward, Berlin September 2016: SlideShare
    • +
  • Till Rohrmann & Fabian Hueske: Declarative stream processing with StreamSQL and CEP Flink Forward, Berlin September 2016: SlideShare
    • +
  • Jamie Grier: Robust Stream Processing with Apache Flink Flink Forward, Berlin September 2016: SlideShare
    • +
  • Jamie Grier: The Stream Processor as a Database- Building Online Applications directly on Streams Flink Forward, Berlin September 2016: SlideShare
    • +
  • Till Rohramnn: Dynamic Scaling - How Apache Flink adapts to changing workloads Flink Forward, Berlin September 2016: SlideShare
    • +
  • Stephan Ewen: Running Flink Everywhere Flink Forward, Berlin September 2016: SlideShare
    • +
  • Stephan Ewen: Scaling Apache Flink to very large State Flink Forward, Berlin September 2016: SlideShare
    • +
  • Aljoscha Krettek: The Future of Apache Flink Flink Forward, Berlin September 2016: SlideShare
    • +
  • Fabian Hueske: Taking a look under the hood of Apache Flink’s relational APIs Flink Forward, Berlin September 2016: SlideShare
    • +
  • Kostas Tzoumas: Streaming in the Wild with Apache Flink Hadoop Summit San Jose, June 2016: SlideShare
    • +
  • Stephan Ewen: The Stream Processor as the Database - Apache Flink Berlin Buzzwords, June 2016: SlideShare
    • +
  • Till Rohrmann & Fabian Hueske: Streaming Analytics & CEP - Two sides of the same coin? Berlin Buzzwords, June 2016: SlideShare
    • +
  • Robert Metzger: A Data Streaming Architecture with Apache Flink Berlin Buzzwords, June 2016: SlideShare
    • +
  • Stephan Ewen: Continuous Processing with Apache Flink Strata + Hadoop World London, May 2016: SlideShare
    • +
  • Stephan Ewen: Streaming Analytics with Apache Flink 1.0 Flink NYC Flink, May 2016: SlideShare
    • +
  • Ufuk Celebi: Unified Stream & Batch Processing with Apache Flink. Hadoop Summit Dublin, April 2016: SlideShare
    • +
  • Kostas Tzoumas: Counting Elements in Streams. Strata San Jose, March 2016: SlideShare
    • +
  • Jamie Grier: Extending the Yahoo! Streaming Benchmark. Flink Washington DC Meetup, March 2016: SlideShare
    • +
  • Jamie Grier: Stateful Stream Processing at In-Memory Speed. Flink NYC Meetup, March 2016: SlideShare
    • +
  • Robert Metzger: Stream Processing with Apache Flink. QCon London, March 2016: SlideShare
    • +
  • Vasia Kalavri: Batch and Stream Graph Processing with Apache Flink. Flink and Neo4j Meetup Berlin, March 2016: SlideShare
    • +
  • Maximilian Michels: Stream Processing with Apache Flink. Big Data Technology Summit, February 2016: +SlideShare
    • +
  • Vasia Kalavri: Single-Pass Graph Streaming Analytics with Apache Flink. FOSDEM, January 2016: SlideShare
    • +
  • Till Rohrmann: Streaming Done Right. FOSDEM, January 2016: SlideShare
    • +
+ +

2015

+ +
    +
  • Till Rohrmann: Streaming Data Flow with Apache Flink (October 29th, 2015): SlideShare
    • +
  • Stephan Ewen: Flink-0.10 (October 28th, 2015): SlideShare
    • +
  • Robert Metzger: Architecture of Flink’s Streaming Runtime (ApacheCon, September 29th, 2015): SlideShare
    • +
  • Robert Metzger: Click-Through Example for Flink’s KafkaConsumer Checkpointing (September, 2015): SlideShare
    • +
  • Paris Carbone: Apache Flink Streaming. Resiliency and Consistency (Google Tech Talk, August 2015: SlideShare
    • +
  • Andra Lungu: Graph Processing with Apache Flink (August 26th, 2015): SlideShare
    • +
  • Till Rohrmann: Interactive data analytisis with Apache Flink (June 23rd, 2015): SlideShare
    • +
  • Gyula Fóra: Real-time data processing with Apache Flink (Budapest Data Forum, June 4th, 2015): SlideShare
    • +
  • Till Rohrmann: Machine Learning with Apache Flink (March 23th, 2015): SlideShare
    • +
  • Marton Balassi: Flink Streaming (February 26th, 2015): SlideShare
    • +
  • Vasia Kalavri: Large-Scale Graph Processing with Apache Flink (FOSDEM, 31st January, 2015): SlideShare
    • +
  • Fabian Hueske: Hadoop Compatibility (January 28th, 2015): SlideShare
    • +
  • Kostas Tzoumas: Apache Flink Overview (January 14th, 2015): SlideShare
    • +
+ +

2014

+ +
    +
  • Kostas Tzoumas: Flink Internals (November 18th, 2014): SlideShare
    • +
  • Marton Balassi & Gyula Fóra: The Flink Big Data Analytics Platform (ApachecCon, November 11th, 2014): SlideShare
    • +
  • Till Rohrmann: Introduction to Apache Flink (October 15th, 2014): SlideShare
    • +
+ + +
+
+ +
+
+ +
+ +
+
+

Copyright © 2014-2016 The Apache Software Foundation. All Rights Reserved.

+

Apache Flink, Apache, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.

+

Privacy Policy · RSS feed

+
+
+
+ + + + + + + + + + + http://git-wip-us.apache.org/repos/asf/flink-web/blob/61adc137/content/usecases.html ---------------------------------------------------------------------- diff --git a/content/usecases.html b/content/usecases.html new file mode 100644 index 0000000..0b4407b --- /dev/null +++ b/content/usecases.html @@ -0,0 +1,218 @@ + + + + + + + + Apache Flink: Flink Use Cases + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + +
+
+
+

Flink Use Cases

+ +

To demonstrate how Flink can be applied to unbounded datasets, here’s a selection of real-word Flink users and problems they’re solving with Flink.

+ +

For more examples, please see the Powered by Flink page.

+ +
    +
  • +

    Optimization of e-commerce search results in real-time: Alibaba’s search infrastructure team uses Flink to update product detail and inventory information in real-time, improving relevance for users.

    +
    • +
  • +

    Stream processing-as-a-service for data science teams: King (the creators of Candy Crush Saga) makes real-time analytics available to its data scientists via a Flink-powered internal platform, dramatically shortening the time to insights from game data.

    +
    • +
  • +

    Network / sensor monitoring and error detection: Bouygues Telecom, one of the largest telecom providers in France, uses Flink to monitor its wired and wireless networks, enabling a rapid response to outages throughout the country.

    +
    • +
  • +

    ETL for business intelligence infrastructure: Zalando uses Flink to transform data for easier loading into its data warehouse, converting complex payloads into relatively simple ones and ensuring that analytics end users have faster access to data.

    +
    • +
+ +

We can tease out common threads from these use cases. Based on the examples above, Flink is well-suited for:

+ +
    +
  1. +

    A variety of (sometimes unreliable) data sources: When data is generated by millions of different users or devices, it’s safe to assume that some events will arrive out of the order they actually occurred–and in the case of more significant upstream failures, some events might come hours later than they’re supposed to. Late data needs to be handled so that results are accurate.

    +
    2. +
  2. +

    Applications with state: When applications become more complex than simple filtering or enhancing of single data records, managing state within these applications (e.g., counters, windows of past data, state machines, embedded databases) becomes hard. Flink provides tools so that state is efficient, fault tolerant, and manageable from the outside so you don’t have to build these capabilities yourself.

    +
    3. +
  3. +

    Data that is processed quickly: There is a focus in these use cases on real-time or near-real-time scenarios, where insights from data should be available at nearly the same moment that the data is generated. Flink is fully capable of meeting these latency requirements when necessary.

    +
    4. +
  4. +

    Data in large volumes: These programs would need to be distributed across many nodes (in some cases, thousands) to support the required scale. Flink can run on large clusters just as seamlessly as it runs on small ones.

    +
    5. +
+ +

And for more user stories, we recommend the sessions from Flink Forward 2016, the annual Flink user conference.

+ + +
+
+ +
+
+ +
+ +
+
+

Copyright © 2014-2016 The Apache Software Foundation. All Rights Reserved.

+

Apache Flink, Apache, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.

+

Privacy Policy · RSS feed

+
+
+
+ + + + + + + + + + +