flink-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From trohrm...@apache.org
Subject [7/7] flink-web git commit: Add CEP blog post
Date Wed, 06 Apr 2016 12:38:46 GMT
Add CEP blog post


Project: http://git-wip-us.apache.org/repos/asf/flink-web/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink-web/commit/ebaf2975
Tree: http://git-wip-us.apache.org/repos/asf/flink-web/tree/ebaf2975
Diff: http://git-wip-us.apache.org/repos/asf/flink-web/diff/ebaf2975

Branch: refs/heads/asf-site
Commit: ebaf2975baef8b42bfdccfd3376732179e349b1f
Parents: 1bfef19
Author: Till Rohrmann <trohrmann@apache.org>
Authored: Wed Apr 6 14:36:15 2016 +0200
Committer: Till Rohrmann <trohrmann@apache.org>
Committed: Wed Apr 6 14:36:15 2016 +0200

----------------------------------------------------------------------
 _posts/2016-04-06-cep-monitoring.md             |  211 ++
 community.md.orig                               |  474 +++
 content/blog/feed.xml                           |  307 +-
 content/blog/index.html                         |   40 +-
 content/blog/page2/index.html                   |   44 +-
 content/blog/page3/index.html                   |   40 +-
 content/blog/page4/index.html                   |   45 +-
 content/blog/page5/index.html                   |   30 +
 .../release_1.0.0-changelog_known_issues.html   |   16 +-
 content/community.html                          |   42 +-
 content/community.md.orig                       |  664 ++++
 content/contribute-code.html                    |   32 +-
 content/contribute-documentation.html           |   14 +-
 content/downloads.html                          |   10 +-
 content/faq.html                                |   56 +-
 content/how-to-contribute.html                  |   32 +-
 content/img/blog/cep-monitoring.svg             | 2838 ++++++++++++++++++
 content/improve-website.html                    |   10 +-
 content/index.html                              |    8 +-
 content/material.html                           |   10 +-
 .../2014/01/13/stratosphere-release-0.4.html    |    6 +-
 .../18/amazon-elastic-mapreduce-cloud-yarn.html |    6 +-
 content/news/2014/11/04/release-0.7.0.html      |    2 +-
 .../news/2014/11/18/hadoop-compatibility.html   |    4 +-
 content/news/2015/01/21/release-0.8.html        |    2 +-
 content/news/2015/02/04/january-in-flink.html   |    2 +-
 content/news/2015/02/09/streaming-example.html  |    4 +-
 .../peeking-into-Apache-Flinks-Engine-Room.html |   18 +-
 .../05/11/Juggling-with-Bits-and-Bytes.html     |   26 +-
 .../news/2015/05/14/Community-update-April.html |    4 +-
 .../2015/08/24/introducing-flink-gelly.html     |   24 +-
 content/news/2015/09/16/off-heap-memory.html    |    2 +-
 content/news/2015/11/16/release-0.10.0.html     |    2 +-
 .../news/2015/12/04/Introducing-windows.html    |   12 +-
 content/news/2016/03/08/release-1.0.0.html      |    4 +-
 content/news/2016/04/06/cep-monitoring.html     |  389 +++
 content/slides.html                             |   12 +-
 img/blog/cep-monitoring.svg                     | 2838 ++++++++++++++++++
 38 files changed, 7972 insertions(+), 308 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flink-web/blob/ebaf2975/_posts/2016-04-06-cep-monitoring.md
----------------------------------------------------------------------
diff --git a/_posts/2016-04-06-cep-monitoring.md b/_posts/2016-04-06-cep-monitoring.md
new file mode 100644
index 0000000..10ca6ee
--- /dev/null
+++ b/_posts/2016-04-06-cep-monitoring.md
@@ -0,0 +1,211 @@
+---
+layout: post
+title: "Introducing Complex Event Processing (CEP) with Apache Flink"
+date: 2016-04-06 10:00:00
+author: "Till Rohrmann"
+author-twitter: "stsffap"
+excerpt: In this blog post, we introduce Flink's new <a href="https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/libs/cep.html">CEP
library</a> that allows you to do pattern matching on event streams. Through the example
of monitoring a data center and generating alerts, we showcase the library's ease of use and
its intuitive Pattern API.
+categories: news
+---
+
+With the ubiquity of sensor networks and smart devices continuously collecting more and more
data, we face the challenge to analyze an ever growing stream of data in near real-time. 
+Being able to react quickly to changing trends or to deliver up to date business intelligence
can be a decisive factor for a company’s success or failure. 
+A key problem in real time processing is the detection of event patterns in data streams.
+
+Complex event processing (CEP) addresses exactly this problem of matching continuously incoming
events against a pattern. 
+The result of a matching are usually complex events which are derived from the input events.

+In contrast to traditional DBMSs where a query is executed on stored data, CEP executes data
on a stored query. 
+All data which is not relevant for the query can be immediately discarded. 
+The advantages of this approach are obvious, given that CEP queries are applied on a potentially
infinite stream of data. 
+Furthermore, inputs are processed immediately. 
+Once the system has seen all events for a matching sequence, results are emitted straight
away. 
+This aspect effectively leads to CEP’s real time analytics capability.
+
+Consequently, CEP’s processing paradigm drew significant interest and found application
in a wide variety of use cases. 
+Most notably, CEP is used nowadays for financial applications such as stock market trend
and credit card fraud detection. 
+Moreover, it is used in RFID-based tracking and monitoring, for example, to detect thefts
in a warehouse where items are not properly checked out. 
+CEP can also be used to detect network intrusion by specifying patterns of suspicious user
behaviour.
+
+Apache Flink with its true streaming nature and its capabilities for low latency as well
as high throughput stream processing is a natural fit for CEP workloads. 
+Consequently, the Flink community has introduced the first version of a new [CEP library](https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/libs/cep.html)
with [Flink 1.0](http://flink.apache.org/news/2016/03/08/release-1.0.0.html). 
+In the remainder of this blog post, we introduce Flink’s CEP library and we illustrate
its ease of use through the example of monitoring a data center.
+
+## Monitoring and alert generation for data centers
+
+<center>
+<img src="{{site.baseurl}}/img/blog/cep-monitoring.svg" style="width:600px;margin:15px">
+</center>
+
+Assume we have a data center with a number of racks. 
+For each rack the power consumption and the temperature are monitored. 
+Whenever such a measurement takes place, a new power or temperature event is generated, respectively.

+Based on this monitoring event stream, we want to detect racks that are about to overheat,
and dynamically adapt their workload and cooling.
+
+For this scenario we use a two staged approach. 
+First, we monitor the temperature events. 
+Whenever we see two consecutive events whose temperature exceeds a threshold value, we generate
a temperature warning with the current average temperature. 
+A temperature warning does not necessarily indicate that a rack is about to overheat. 
+But whenever we see two consecutive warnings with increasing temperatures, then we want to
issue an alert for this rack. 
+This alert can then lead to countermeasures to cool the rack.
+
+### Implementation with Apache Flink
+
+First, we define the messages of the incoming monitoring event stream. 
+Every monitoring message contains its originating rack ID. 
+The temperature event additionally contains the current temperature and the power consumption
event contains the current voltage. 
+We model the events as POJOs:
+
+```java
+public abstract class MonitoringEvent {
+    private int rackID;
+    ...
+}
+
+public class TemperatureEvent extends MonitoringEvent {
+    private double temperature;
+    ...
+}
+
+public class PowerEvent extends MonitoringEvent {
+    private double voltage;
+    ...
+}
+```
+
+Now we can ingest the monitoring event stream using one of Flink’s connectors (e.g. Kafka,
RabbitMQ, etc.). 
+This will give us a `DataStream<MonitoringEvent> inputEventStream` which we will use
as the input for Flink’s CEP operator. 
+But first, we have to define the event pattern to detect temperature warnings. 
+The CEP library offers an intuitive [Pattern API](https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/libs/cep.html#the-pattern-api)
to easily define these complex patterns.
+
+Every pattern consists of a sequence of events which can have optional filter conditions
assigned. 
+A pattern always starts with a first event to which we will assign the name `“First Event”`.
+
+```java
+Pattern.<MonitoringEvent>begin("First Event");
+```
+
+This pattern will match every monitoring event. 
+Since we are only interested in `TemperatureEvents` whose temperature is above a threshold
value, we have to add an additional subtype constraint and a where clause:
+
+```java
+Pattern.<MonitoringEvent>begin("First Event")
+    .subtype(TemperatureEvent.class)
+    .where(evt -> evt.getTemperature() >= TEMPERATURE_THRESHOLD);
+```
+
+As stated before, we want to generate a `TemperatureWarning` if and only if we see two consecutive
`TemperatureEvents` for the same rack whose temperatures are too high. 
+The Pattern API offers the `next` call which allows us to add a new event to our pattern.

+This event has to follow directly the first matching event in order for the whole pattern
to match.
+
+```java
+Pattern<MonitoringEvent, ?> warningPattern = Pattern.<MonitoringEvent>begin("First
Event")
+    .subtype(TemperatureEvent.class)
+    .where(evt -> evt.getTemperature() >= TEMPERATURE_THRESHOLD)
+    .next("Second Event")
+    .subtype(TemperatureEvent.class)
+    .where(evt -> evt.getTemperature() >= TEMPERATURE_THRESHOLD)
+    .within(Time.seconds(10));
+```
+
+The final pattern definition also contains the `within` API call which defines that two consecutive
`TemperatureEvents` have to occur within a time interval of 10 seconds for the pattern to
match. 
+Depending on the time characteristic setting, this can either be processing, ingestion or
event time.
+
+Having defined the event pattern, we can now apply it on the `inputEventStream`.
+
+```java
+PatternStream<MonitoringEvent> tempPatternStream = CEP.pattern(
+    inputEventStream.keyBy("rackID"),
+    warningPattern);
+```
+
+Since we want to generate our warnings for each rack individually, we `keyBy` the input event
stream by the `“rackID”` POJO field. 
+This enforces that matching events of our pattern will all have the same rack ID.
+
+The `PatternStream<MonitoringEvent>` gives us access to successfully matched event
sequences. 
+They can be accessed using the `select` API call. 
+The `select` API call takes a `PatternSelectFunction` which is called for every matching
event sequence. 
+The event sequence is provided as a `Map<String, MonitoringEvent>` where each `MonitoringEvent`
is identified by its assigned event name. 
+Our pattern select function generates for each matching pattern a `TemperatureWarning` event.
+
+```java
+public class TemperatureWarning {
+    private int rackID;
+    private double averageTemperature;
+    ...
+}
+
+DataStream<TemperatureWarning> warnings = tempPatternStream.select(
+    (Map<String, MonitoringEvent> pattern) -> {
+        TemperatureEvent first = (TemperatureEvent) pattern.get("First Event");
+        TemperatureEvent second = (TemperatureEvent) pattern.get("Second Event");
+
+        return new TemperatureWarning(
+            first.getRackID(), 
+            (first.getTemperature() + second.getTemperature()) / 2);
+    }
+);
+```
+
+Now we have generated a new complex event stream `DataStream<TemperatureWarning> warnings`
from the initial monitoring event stream. 
+This complex event stream can again be used as the input for another round of complex event
processing. 
+We use the `TemperatureWarnings` to generate `TemperatureAlerts` whenever we see two consecutive
`TemperatureWarnings` for the same rack with increasing temperatures. 
+The `TemperatureAlerts` have the following definition:
+
+```java
+public class TemperatureAlert {
+    private int rackID;
+    ...
+}
+```
+
+At first, we have to define our alert event pattern:
+
+```java
+Pattern<TemperatureWarning, ?> alertPattern = Pattern.<TemperatureWarning>begin("First
Event")
+    .next("Second Event")
+    .within(Time.seconds(20));
+```
+
+This definition says that we want to see two `TemperatureWarnings` within 20 seconds. 
+The first event has the name `“First Event”` and the second consecutive event has the
name `“Second Event”`. 
+The individual events don’t have a where clause assigned, because we need access to both
events in order to decide whether the temperature is increasing. 
+Therefore, we apply the filter condition in the select clause. 
+But first, we obtain again a `PatternStream`.
+
+```java
+PatternStream<TemperatureWarning> alertPatternStream = CEP.pattern(
+    warnings.keyBy("rackID"),
+    alertPattern);
+```
+
+Again, we `keyBy` the warnings input stream by the `"rackID"` so that we generate our alerts
for each rack individually. 
+Next we apply the `flatSelect` method which will give us access to matching event sequences
and allows us to output an arbitrary number of complex events. 
+Thus, we will only generate a `TemperatureAlert` if and only if the temperature is increasing.
+
+```java
+DataStream<TemperatureAlert> alerts = alertPatternStream.flatSelect(
+    (Map<String, TemperatureWarning> pattern, Collector<TemperatureAlert> out)
-> {
+        TemperatureWarning first = pattern.get("First Event");
+        TemperatureWarning second = pattern.get("Second Event");
+
+        if (first.getAverageTemperature() < second.getAverageTemperature()) {
+            out.collect(new TemperatureAlert(first.getRackID()));
+        }
+    });
+```
+
+The `DataStream<TemperatureAlert> alerts` is the data stream of temperature alerts
for each rack. 
+Based on these alerts we can now adapt the workload or cooling for overheating racks.
+
+The full source code for the presented example as well as an example data source which generates
randomly monitoring events can be found in [this repository](https://github.com/tillrohrmann/cep-monitoring).
+
+## Conclusion
+
+In this blog post we have seen how easy it is to reason about event streams using Flink’s
CEP library. 
+Using the example of monitoring and alert generation for a data center, we have implemented
a short program which notifies us when a rack is about to overheat and potentially to fail.
+
+In the future, the Flink community will further extend the CEP library’s functionality
and expressiveness. 
+Next on the road map is support for a regular expression-like pattern specification, including
Kleene star, lower and upper bounds, and negation. 
+Furthermore, it is planned to allow the where-clause to access fields of previously matched
events. 
+This feature will allow to prune unpromising event sequences early.
+

http://git-wip-us.apache.org/repos/asf/flink-web/blob/ebaf2975/community.md.orig
----------------------------------------------------------------------
diff --git a/community.md.orig b/community.md.orig
new file mode 100644
index 0000000..d3ec5f8
--- /dev/null
+++ b/community.md.orig
@@ -0,0 +1,474 @@
+---
+title: "Community"
+---
+
+There are many ways to get help from the Apache Flink community. The [mailing lists](#mailing-lists)
are the primary place where all Flink committers are present. If you want to talk with the
Flink committers and users in a chat, there is a [IRC channel](#irc). Some committers are
also monitoring [Stack Overflow](http://stackoverflow.com/questions/tagged/flink). Please
remember to tag your questions with the *[flink](http://stackoverflow.com/questions/tagged/flink)*
tag. Bugs and feature requests can either be discussed on *dev mailing list* or on [JIRA]({{
site.jire }}). Those interested in contributing to Flink should check out the [contribution
guide](how-to-contribute.html).
+
+{% toc %}
+
+## Mailing Lists
+
+<table class="table table-striped">
+  <thead>
+    <th class="text-center">Name</th>
+    <th class="text-center">Subscribe</th>
+    <th class="text-center">Digest</th>
+    <th class="text-center">Unsubscribe</th>
+    <th class="text-center">Post</th>
+    <th class="text-center">Archive</th>
+  </thead>
+  <tr>
+    <td>
+      <strong>news</strong>@flink.apache.org<br>
+      <small>News and announcements from the Flink community.</small>
+    </td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:news-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:news-digest-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:news-unsubscribe@flink.apache.org">Unsubscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <i>Read
only list</i></td>
+    <td class="text-center">
+      <a href="http://mail-archives.apache.org/mod_mbox/flink-news/">Archives</a>
<br>
+    </td>
+  </tr>
+  <tr>
+    <td>
+      <strong>user</strong>@flink.apache.org<br>
+      <small>User support and questions mailing list</small>
+    </td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:user-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:user-digest-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:user-unsubscribe@flink.apache.org">Unsubscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:user@flink.apache.org">Post</a></td>
+    <td class="text-center">
+      <a href="http://mail-archives.apache.org/mod_mbox/flink-user/">Archives</a>
<br>
+      <a href="http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/">Nabble
Archive</a>
+    </td>
+  </tr>
+  <tr>
+    <td>
+      <strong>dev</strong>@flink.apache.org<br>
+      <small>Development related discussions</small>
+    </td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:dev-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:dev-digest-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:dev-unsubscribe@flink.apache.org">Unsubscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:dev@flink.apache.org">Post</a></td>
+    <td class="text-center">
+      <a href="http://mail-archives.apache.org/mod_mbox/flink-dev/">Archives</a>
<br>
+      <a href="http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/">Nabble
Archive</a>
+    </td>
+  </tr>
+  <tr>
+    <td>
+      <strong>issues</strong>@flink.apache.org
+      <br>
+      <small>Mirror of all JIRA activity</small>
+    </td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:issues-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:issues-digest-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:issues-unsubscribe@flink.apache.org">Unsubscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i><i>Read
only list</i></td>
+    <td class="text-center"><a href="http://mail-archives.apache.org/mod_mbox/flink-issues/">Archives</a></td>
+  </tr>
+  <tr>
+    <td>
+      <strong>commits</strong>@flink.apache.org
+      <br>
+      <small>All commits to our repositories</small>
+    </td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:commits-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:commits-digest-subscribe@flink.apache.org">Subscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <a
href="mailto:commits-unsubscribe@flink.apache.org">Unsubscribe</a></td>
+    <td class="text-center"><i class="fa fa-pencil-square-o"></i> <i>Read
only list</i></td>
+    <td class="text-center"><a href="http://mail-archives.apache.org/mod_mbox/flink-commits/">Archives</a></td>
+  </tr>
+</table>
+
+## IRC
+
+There is an IRC channel called #flink dedicated to Apache Flink at irc.freenode.org. There
is also a [web-based IRC client](http://webchat.freenode.net/?channels=flink) available.
+
+The IRC channel can be used for online discussions about Apache Flink as community, but developers
should be careful to move or duplicate all the official or useful discussions to the issue
tracking system or dev mailing list.
+
+## Stack Overflow
+
+Committers are watching [Stack Overflow](http://stackoverflow.com/questions/tagged/flink)
for the [flink](http://stackoverflow.com/questions/tagged/flink) tag.
+
+Make sure to tag your questions there accordingly to get answers from the Flink community.
+
+## Issue Tracker
+
+We use JIRA to track all code related issues: [{{ site.jira }}]({{ site.jira }}).
+
+All issue activity is also mirrored to the issues mailing list.
+
+## Source Code
+
+### Main source repositories
+
+- **ASF writable**: [https://git-wip-us.apache.org/repos/asf/flink.git](https://git-wip-us.apache.org/repos/asf/flink.git)
+- **ASF read-only**: git://git.apache.org/repos/asf/flink.git
+- **GitHub mirror**: [https://github.com/apache/flink.git](https://github.com/apache/flink.git)
+
+Note: Flink does not build with Oracle JDK 6. It runs with Oracle JDK 6.
+
+### Website repositories
+
+- **ASF writable**: [https://git-wip-us.apache.org/repos/asf/flink-web.git](https://git-wip-us.apache.org/repos/asf/flink-web.git)
+- **ASF read-only**: git://git.apache.org/repos/asf/flink-web.git
+- **GitHub mirror**:  [https://github.com/apache/flink-web.git](https://github.com/apache/flink-web.git)
+
+
+# Third party packages
+
+This is a list of third party packages (ie, libraries, system extensions, or examples) built
on Flink.
+The Flink community only collects links to those packages but does not maintain them.
+Thus, they do not belong to the Apache Flink project and the community cannot give any support
for them.
+**Your project is missing?**
+Let us know per [user/dev mailing list](#mailing-lists).
+
+<table class="flink-packages" width="100%">
+  <style>
+  table.flink-packages {
+    table-layout: fixed;
+    border-color: transparent;
+    border-spacing: 5px;
+    border-collapse: separate;
+  }
+  td.flink-package-empty{
+    height: 12em;
+    vertical-align: bottom;
+    border-color: transparent;
+    background-color: transparent;
+  }
+  td.flink-package {
+    height: 12em;
+    vertical-align: bottom;
+    border-color: transparent;
+    background-color: #B9BFC7;
+    color: #E5EAFA;
+  }
+  td.flink-package-external {
+    height: 12em;
+    vertical-align: bottom;
+    border-color: transparent;
+    background-color: #F5A030;
+  }
+  td.flink-package > div {
+    transform: rotate(-90deg);
+    font-size: 12pt;
+    white-space: nowrap;
+  }
+  td.flink-package-external > div {
+    transform: rotate(-90deg);
+    font-size: 16pt;
+    white-space: nowrap;
+  }
+  td.flink-component {
+    font-size: 16pt;
+    text-align: center;
+    border-color: transparent;
+    color: #E5EAFA;
+  }
+  </style>
+  <tr>
+   <td class="flink-package"><div><span>&nbsp;Flink ML</span></div></td>
+   <td class="flink-package"><div><span>&nbsp;Table API</span></div></td>
+   <td class="flink-package"><div><span>&nbsp;Gelly (Graph)</span></div></td>
+   <td class="flink-package"><div><span>&nbsp;MapReduce</span></div></td>
+   <td class="flink-package"><div><span>&nbsp;Python</span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#apache-zeppelin">Apache
Zeppelin</a></span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#apache-mahout">Apache
Mahout</a></span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#cascading">Cascading</a></span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#dataflow">Dataflow</a></span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#gradoop">Gradoop</a></span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#bigpetstore">BigPetStore</a></span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#fastr">FastR</a></span></div></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package"><div><span>&nbsp;Table API</span></div></td>
+   <td class="flink-package"><div><span>&nbsp;Storm</span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#apache-samoa">Apache
SAMOA</a></span></div></td>
+   <td class="flink-package-external"><div><span>&nbsp;<a href="#dataflow">Dataflow</a></span></div></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+   <td class="flink-package-empty"></td>
+  </tr>
+
+  <tr bgcolor="#B9BFC7">
+   <td class="flink-component" colspan="15">DataSet API</td>
+   <td class="flink-component" colspan="15">DataStream API</td>
+  </tr>
+
+  <tr bgcolor="#B9BFC7">
+   <td class="flink-component" colspan="30">Flink Runtime</td>
+  </tr>
+
+  <tr bgcolor="#B9BFC7">
+   <td class="flink-component" colspan="6" width="20%">Local</td>
+   <td class="flink-component" colspan="6" width="20%">Remote</td>
+   <td class="flink-component" colspan="6" width="20%">Embedded</td>
+   <td class="flink-component" colspan="6" width="20%">YARN</td>
+   <td class="flink-component" colspan="6" width="20%">Tez</td>
+  </tr>
+</table>
+
+## Apache Zeppelin
+
+[Apache Zeppelin (incubator)](https://zeppelin.incubator.apache.org/) is a web-based notebook
that enables interactive data analytics and can be used with 
+[Flink as an execution engine](https://zeppelin.incubator.apache.org/docs/interpreter/flink.html)
(next to others engines).
+See also Jim Dowling's [Flink Forward talk](http://www.slideshare.net/FlinkForward/jim-dowling-interactive-flink-analytics-with-hopsworks-and-zeppelin)
about Zeppelin on Flink.
+
+## Apache Mahout 
+
+[Apache Mahout](https://mahout.apache.org/) in a machine learning library that will feature
Flink as an execution engine soon.
+Check out Sebastian Schelter's [Flink Forward talk](http://www.slideshare.net/FlinkForward/sebastian-schelter-distributed-machine-learing-with-the-samsara-dsl)
about Mahout-Samsara DSL.
+
+## Cascading 
+
+[Cascading](http://www.cascading.org/cascading-flink/) enables an user to build complex workflows
easily on Flink and other execution engines.
+[Cascading on Flink](https://github.com/dataArtisans/cascading-flink) is build by [dataArtisans](http://data-artisans.com/)
and [Driven, Inc](http://www.driven.io/).
+See Fabian Hueske's [Flink Forward talk](http://www.slideshare.net/FlinkForward/fabian-hueske-training-cascading-on-flink)
for more details.
+
+## Google Dataflow
+
+Currently, Google Dataflow programs can be executed in batch mode on top of Flink via [FlinkPipelineRunner](https://github.com/dataArtisans/flink-dataflow).
+Executing streaming programs is WIP by [dataArtisans](http://data-artisans.com/).
+Maximilian Michels' [Flink Forward talk](http://www.slideshare.net/FlinkForward/maximilian-michels-google-cloud-dataflow-on-top-of-apache-flink)
gives a good introduction.
+
+## GRADOOP
+
+[GRADOOP](http://dbs.uni-leipzig.de/en/research/projects/gradoop) enables scalable graph
analytics on top of Flink and is developed at Leipzig University. Check out [Martin Junghanns’
Flink Forward talk](http://www.slideshare.net/FlinkForward/martin-junghans-gradoop-scalable-graph-analytics-with-apache-flink).
+
+## BigPetStore
+
+[BigPetStore](https://github.com/apache/bigtop/tree/master/bigtop-bigpetstore) is a benchmarking
suite including a data generator and will be available for Flin k soon.
+See Suneel Marthi's [Flink Forward talk](http://www.slideshare.net/FlinkForward/suneel-marthi-bigpetstore-flink-a-comprehensive-blueprint-for-apache-flink?ref=http://flink-forward.org/?session=tbd-3)
as preview.
+
+## FastR
+
+[FastR](https://bitbucket.org/allr/fastr-flink) in an implemenation of the R language in
Java. [FastR Flink](https://bitbucket.org/allr/fastr-flink/src/3535a9b7c7f208508d6afbcdaf1de7d04fa2bf79/README_FASTR_FLINK.md?at=default&fileviewer=file-view-default)
exeutes R workload on top of Flink.
+
+## Apache SAMOA
+
+[Apache SAMOA (incubating)](https://samoa.incubator.apache.org/) a streaming ML library featuring
Flink an execution engine soon. Albert Bifet introduced SAMOA on Flink at his [Flink Forward
talk](http://www.slideshare.net/FlinkForward/albert-bifet-apache-samoa-mining-big-data-streams-with-apache-flink?ref=http://flink-forward.org/?session=apache-samoa-mining-big-data-streams-with-apache-flink).
+
+## Python Examples on Flink
+
+A [collection of examples](https://github.com/wdm0006/flink-python-examples) using Apache
Flink's Python API.
+
+## WordCount Example in Clojure
+
+Small [WordCount example](https://github.com/mjsax/flink-external/tree/master/flink-clojure)
on how to write a Flink program in Clojure.
+
+## Anomaly Detection and Prediction in Flink
+
+[flink-htm](https://github.com/nupic-community/flink-htm) is a library for anomaly detection
and prediction in Apache Flink. The algorithms are based on Hierarchical Temporal Memory (HTM)
as implemented by the Numenta Platform for Intelligent Computing (NuPIC).
+
+# People
+
+<table class="table table-striped">
+  <thead>
+    <th class="text-center"></th>
+    <th class="text-center">Name</th>
+    <th class="text-center">Role</th>
+    <th class="text-center">Apache ID</th>
+  </thead>
+  <tr>
+    <td class="text-center"><img src="https://avatars2.githubusercontent.com/u/5990983?s=50"></td>
+    <td class="text-center">Márton Balassi</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">mbalassi</td>
+  </tr>
+    <tr>
+        <td class="text-center"><img src="https://avatars2.githubusercontent.com/u/858078?v=3&s=50"></td>
+        <td class="text-center">Paris Carbone</td>
+        <td class="text-center">Committer</td>
+        <td class="text-center">senorcarbone</td>
+    </tr>
+  <tr>
+    <td class="text-center" width="10%"><img src="https://avatars3.githubusercontent.com/u/1756620?s=50"></a></td>
+    <td class="text-center">Ufuk Celebi</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">uce</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars2.githubusercontent.com/u/1727146?s=50"></td>
+    <td class="text-center">Stephan Ewen</td>
+    <td class="text-center">PMC, Committer, VP</td>
+    <td class="text-center">sewen</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars1.githubusercontent.com/u/5880972?s=50"></td>
+    <td class="text-center">Gyula Fóra</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">gyfora</td>
+  </tr>
+  <tr>
+    <td class="text-center"></td>
+    <td class="text-center">Alan Gates</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">gates</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars0.githubusercontent.com/u/2388347?s=50"></td>
+    <td class="text-center">Fabian Hueske</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">fhueske</td>
+  </tr>
+    <tr>
+    <td class="text-center"><img src="https://avatars3.githubusercontent.com/u/498957?v=3&s=50"></td>
+    <td class="text-center">Vasia Kalavri</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">vasia</td>
+  </tr>
+  </tr>
+    <tr>
+    <td class="text-center"><img src="https://avatars0.githubusercontent.com/u/68551?s=50"></td>
+    <td class="text-center">Aljoscha Krettek</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">aljoscha</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars2.githubusercontent.com/u/2550549?s=50"></td>
+    <td class="text-center">Andra Lungu</td>
+    <td class="text-center">Committer</td>
+    <td class="text-center">andra</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars0.githubusercontent.com/u/89049?s=50"></td>
+    <td class="text-center">Robert Metzger</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">rmetzger</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars2.githubusercontent.com/u/837221?s=50"></td>
+    <td class="text-center">Maximilian Michels</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">mxm</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars2.githubusercontent.com/u/1941681?s=50"></td>
+    <td class="text-center">Chiwan Park</td>
+    <td class="text-center">Committer</td>
+    <td class="text-center">chiwanpark</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars1.githubusercontent.com/u/5756858?s=50"></td>
+    <td class="text-center">Till Rohrmann</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">trohrmann</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars0.githubusercontent.com/u/105434?s=50"></td>
+    <td class="text-center">Henry Saputra</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">hsaputra</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars0.githubusercontent.com/u/8959638?s=50"></td>
+    <td class="text-center">Matthias J. Sax</td>
+    <td class="text-center">Committer</td>
+    <td class="text-center">mjsax</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars1.githubusercontent.com/u/409707?s=50"></td>
+    <td class="text-center">Sebastian Schelter</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">ssc</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars2.githubusercontent.com/u/1925554?s=50"></td>
+    <td class="text-center">Kostas Tzoumas</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">ktzoumas</td>
+  </tr>
+  <tr>
+    <td class="text-center"></td>
+    <td class="text-center">Timo Walther</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">twalthr</td>
+  </tr> 
+  <tr>
+    <td class="text-center"><img src="https://avatars1.githubusercontent.com/u/1826769?s=50"></td>
+    <td class="text-center">Daniel Warneke</td>
+    <td class="text-center">PMC, Committer</td>
+    <td class="text-center">warneke</td>
+  </tr>
+  <tr>
+    <td class="text-center"><img src="https://avatars1.githubusercontent.com/u/4425616?s=50"></td>
+    <td class="text-center">ChengXiang Li</td>
+    <td class="text-center">Committer</td>
+    <td class="text-center">chengxiang</td>
+  </tr>
+<<<<<<< 1bfef19c46ea9fb303a717d6893499c4a1afafb5
+  <tr>
+    <td class="text-center"><img src="https://avatars1.githubusercontent.com/u/569655?s=50"></td>
+    <td class="text-center">Greg Hogan</td>
+    <td class="text-center">Committer</td>
+    <td class="text-center">greg</td>
+  </tr>
+=======
+>>>>>>> Add chengxiang to the People list.
+</table>
+
+You can reach committers directly at `<apache-id>@apache.org`. A list of all contributors
can be found [here]({{ site.FLINK_CONTRIBUTORS_URL }}).
+
+## Former mentors
+
+The following people were very kind to mentor the project while in incubation.
+
+<table class="table table-striped">
+  <thead>
+    <th class="text-center"></th>
+    <th class="text-center">Name</th>
+    <th class="text-center">Role</th>
+    <th class="text-center">Apache ID</th>
+  </thead>
+  <tr>
+    <td class="text-center"></td>
+    <td class="text-center">Ashutosh Chauhan</td>
+    <td class="text-center">Former PPMC, Mentor</td>
+    <td class="text-center">hashutosh</td>
+  </tr>
+  <tr>
+    <td class="text-center"></td>
+    <td class="text-center">Ted Dunning</td>
+    <td class="text-center">Former PPMC, Mentor</td>
+    <td class="text-center">tdunning</td>
+  </tr>
+  <tr>
+    <td class="text-center"></td>
+    <td class="text-center">Alan Gates</td>
+    <td class="text-center">Former PPMC, Mentor</td>
+    <td class="text-center">gates</td>
+  </tr>
+  </tr>
+    <tr>
+    <td class="text-center"></td>
+    <td class="text-center">Owen O'Malley</td>
+    <td class="text-center">Former PPMC, Mentor</td>
+    <td class="text-center">omalley</td>
+  </tr>
+  <tr>
+    <td class="text-center"></td>
+    <td class="text-center">Sean Owen</td>
+    <td class="text-center">Former PPMC, Mentor</td>
+    <td class="text-center">srowen</td>
+  </tr>
+  <tr>
+    <td class="text-center"></td>
+    <td class="text-center">Henry Saputra</td>
+    <td class="text-center">Former PPMC, Mentor</td>
+    <td class="text-center">hsaputra</td>
+  </tr>
+</table>


Mime
View raw message