flink-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ktzou...@apache.org
Subject [1/4] flink-web git commit: Added blog post about 2015: a year in review
Date Fri, 18 Dec 2015 15:33:46 GMT
Repository: flink-web
Updated Branches:
  refs/heads/asf-site 1e63786d3 -> b90ef37c5


http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/content/news/2015/12/11/storm-compatibility.html
----------------------------------------------------------------------
diff --git a/content/news/2015/12/11/storm-compatibility.html b/content/news/2015/12/11/storm-compatibility.html
index 1ec4470..04503b0 100644
--- a/content/news/2015/12/11/storm-compatibility.html
+++ b/content/news/2015/12/11/storm-compatibility.html
@@ -198,13 +198,13 @@ For this, you only need to replace the dependency <code>storm-core</code>
by <co
 First, the program is assembled the Storm way without any code change to Spouts, Bolts, or
the topology itself.</p>
 
 <div class="highlight"><pre><code class="language-java"><span class="c1">//
assemble topology, the Storm way</span>
-<span class="n">TopologyBuilder</span> <span class="n">builder</span>
<span class="o">=</span> <span class="k">new</span> <span class="n">TopologyBuilder</span><span
class="o">();</span>
-<span class="n">builder</span><span class="o">.</span><span class="na">setSpout</span><span
class="o">(</span><span class="s">&quot;source&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="n">StormFileSpout</span><span
class="o">(</span><span class="n">inputFilePath</span><span class="o">));</span>
-<span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span
class="o">(</span><span class="s">&quot;tokenizer&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="n">StormBoltTokenizer</span><span
class="o">())</span>
+<span class="n">TopologyBuilder</span> <span class="n">builder</span>
<span class="o">=</span> <span class="k">new</span> <span class="nf">TopologyBuilder</span><span
class="o">();</span>
+<span class="n">builder</span><span class="o">.</span><span class="na">setSpout</span><span
class="o">(</span><span class="s">&quot;source&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="nf">StormFileSpout</span><span
class="o">(</span><span class="n">inputFilePath</span><span class="o">));</span>
+<span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span
class="o">(</span><span class="s">&quot;tokenizer&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="nf">StormBoltTokenizer</span><span
class="o">())</span>
        <span class="o">.</span><span class="na">shuffleGrouping</span><span
class="o">(</span><span class="s">&quot;source&quot;</span><span
class="o">);</span>
-<span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span
class="o">(</span><span class="s">&quot;counter&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="n">StormBoltCounter</span><span
class="o">())</span>
-       <span class="o">.</span><span class="na">fieldsGrouping</span><span
class="o">(</span><span class="s">&quot;tokenizer&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="n">Fields</span><span
class="o">(</span><span class="s">&quot;word&quot;</span><span
class="o">));</span>
-<span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span
class="o">(</span><span class="s">&quot;sink&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="n">StormBoltFileSink</span><span
class="o">(</span><span class="n">outputFilePath</span><span class="o">))</span>
+<span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span
class="o">(</span><span class="s">&quot;counter&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="nf">StormBoltCounter</span><span
class="o">())</span>
+       <span class="o">.</span><span class="na">fieldsGrouping</span><span
class="o">(</span><span class="s">&quot;tokenizer&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="nf">Fields</span><span
class="o">(</span><span class="s">&quot;word&quot;</span><span
class="o">));</span>
+<span class="n">builder</span><span class="o">.</span><span class="na">setBolt</span><span
class="o">(</span><span class="s">&quot;sink&quot;</span><span
class="o">,</span> <span class="k">new</span> <span class="nf">StormBoltFileSink</span><span
class="o">(</span><span class="n">outputFilePath</span><span class="o">))</span>
        <span class="o">.</span><span class="na">shuffleGrouping</span><span
class="o">(</span><span class="s">&quot;counter&quot;</span><span
class="o">);</span></code></pre></div>
 
 <p>In order to execute the topology, we need to translate it to a <code>FlinkTopology</code>
and submit it to a local or remote Flink cluster, very similar to submitting the application
to a Storm cluster.<sup><a href="#fn1" id="ref1">1</a></sup></p>
@@ -213,7 +213,7 @@ First, the program is assembled the Storm way without any code change
to Spouts,
 <span class="c1">// replaces: StormTopology topology = builder.createTopology();</span>
 <span class="n">FlinkTopology</span> <span class="n">topology</span>
<span class="o">=</span> <span class="n">FlinkTopology</span><span
class="o">.</span><span class="na">createTopology</span><span class="o">(</span><span
class="n">builder</span><span class="o">);</span>
 
-<span class="n">Config</span> <span class="n">conf</span> <span
class="o">=</span> <span class="k">new</span> <span class="n">Config</span><span
class="o">();</span>
+<span class="n">Config</span> <span class="n">conf</span> <span
class="o">=</span> <span class="k">new</span> <span class="nf">Config</span><span
class="o">();</span>
 <span class="k">if</span><span class="o">(</span><span class="n">runLocal</span><span
class="o">)</span> <span class="o">{</span>
 	<span class="c1">// use FlinkLocalCluster instead of LocalCluster</span>
 	<span class="n">FlinkLocalCluster</span> <span class="n">cluster</span>
<span class="o">=</span> <span class="n">FlinkLocalCluster</span><span
class="o">.</span><span class="na">getLocalCluster</span><span class="o">();</span>
@@ -253,14 +253,14 @@ As Storm is type agnostic, it is required to specify the output type
of embedded
 <span class="c1">// use Spout as source</span>
 <span class="n">DataStream</span><span class="o">&lt;</span><span
class="n">Tuple1</span><span class="o">&lt;</span><span class="n">String</span><span
class="o">&gt;&gt;</span> <span class="n">source</span> <span
class="o">=</span> 
   <span class="n">env</span><span class="o">.</span><span class="na">addSource</span><span
class="o">(</span><span class="c1">// Flink provided wrapper including original
Spout</span>
-                <span class="k">new</span> <span class="n">SpoutWrapper</span><span
class="o">&lt;</span><span class="n">String</span><span class="o">&gt;(</span><span
class="k">new</span> <span class="n">FileSpout</span><span class="o">(</span><span
class="n">localFilePath</span><span class="o">)),</span> 
+                <span class="k">new</span> <span class="n">SpoutWrapper</span><span
class="o">&lt;</span><span class="n">String</span><span class="o">&gt;(</span><span
class="k">new</span> <span class="nf">FileSpout</span><span class="o">(</span><span
class="n">localFilePath</span><span class="o">)),</span> 
                 <span class="c1">// specify output type manually</span>
                 <span class="n">TypeExtractor</span><span class="o">.</span><span
class="na">getForObject</span><span class="o">(</span><span class="k">new</span>
<span class="n">Tuple1</span><span class="o">&lt;</span><span
class="n">String</span><span class="o">&gt;(</span><span class="s">&quot;&quot;</span><span
class="o">)));</span>
 <span class="c1">// FileSpout cannot be parallelized</span>
 <span class="n">DataStream</span><span class="o">&lt;</span><span
class="n">Tuple1</span><span class="o">&lt;</span><span class="n">String</span><span
class="o">&gt;&gt;</span> <span class="n">text</span> <span
class="o">=</span> <span class="n">source</span><span class="o">.</span><span
class="na">setParallelism</span><span class="o">(</span><span class="mi">1</span><span
class="o">);</span>
 
 <span class="c1">// further processing with Flink</span>
-<span class="n">DataStream</span><span class="o">&lt;</span><span
class="n">Tuple2</span><span class="o">&lt;</span><span class="n">String</span><span
class="o">,</span><span class="n">Integer</span><span class="o">&gt;</span>
<span class="n">tokens</span> <span class="o">=</span> <span class="n">text</span><span
class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span
class="k">new</span> <span class="n">Tokenizer</span><span class="o">()).</span><span
class="na">keyBy</span><span class="o">(</span><span class="mi">0</span><span
class="o">);</span>
+<span class="n">DataStream</span><span class="o">&lt;</span><span
class="n">Tuple2</span><span class="o">&lt;</span><span class="n">String</span><span
class="o">,</span><span class="n">Integer</span><span class="o">&gt;</span>
<span class="n">tokens</span> <span class="o">=</span> <span class="n">text</span><span
class="o">.</span><span class="na">flatMap</span><span class="o">(</span><span
class="k">new</span> <span class="nf">Tokenizer</span><span class="o">()).</span><span
class="na">keyBy</span><span class="o">(</span><span class="mi">0</span><span
class="o">);</span>
 
 <span class="c1">// use Bolt for counting</span>
 <span class="n">DataStream</span><span class="o">&lt;</span><span
class="n">Tuple2</span><span class="o">&lt;</span><span class="n">String</span><span
class="o">,</span><span class="n">Integer</span><span class="o">&gt;</span>
<span class="n">counts</span> <span class="o">=</span>
@@ -268,7 +268,7 @@ As Storm is type agnostic, it is required to specify the output type of
embedded
                    <span class="c1">// specify output type manually</span>
                    <span class="n">TypeExtractor</span><span class="o">.</span><span
class="na">getForObject</span><span class="o">(</span><span class="k">new</span>
<span class="n">Tuple2</span><span class="o">&lt;</span><span
class="n">String</span><span class="o">,</span><span class="n">Integer</span><span
class="o">&gt;(</span><span class="s">&quot;&quot;</span><span
class="o">,</span><span class="mi">0</span><span class="o">))</span>
                    <span class="c1">// Flink provided wrapper including original Bolt</span>
-                   <span class="k">new</span> <span class="n">BoltWrapper</span><span
class="o">&lt;</span><span class="n">String</span><span class="o">,</span><span
class="n">Tuple2</span><span class="o">&lt;</span><span class="n">String</span><span
class="o">,</span><span class="n">Integer</span><span class="o">&gt;&gt;(</span><span
class="k">new</span> <span class="n">BoltCounter</span><span class="o">()));</span>
+                   <span class="k">new</span> <span class="n">BoltWrapper</span><span
class="o">&lt;</span><span class="n">String</span><span class="o">,</span><span
class="n">Tuple2</span><span class="o">&lt;</span><span class="n">String</span><span
class="o">,</span><span class="n">Integer</span><span class="o">&gt;&gt;(</span><span
class="k">new</span> <span class="nf">BoltCounter</span><span class="o">()));</span>
 
 <span class="c1">// write result to file via Flink sink</span>
 <span class="n">counts</span><span class="o">.</span><span class="na">writeAsText</span><span
class="o">(</span><span class="n">outputPath</span><span class="o">);</span>

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/content/news/2015/12/18/a-year-in-review.html
----------------------------------------------------------------------
diff --git a/content/news/2015/12/18/a-year-in-review.html b/content/news/2015/12/18/a-year-in-review.html
new file mode 100644
index 0000000..ba25e8f
--- /dev/null
+++ b/content/news/2015/12/18/a-year-in-review.html
@@ -0,0 +1,423 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <!-- The above 3 meta tags *must* come first in the head; any other head content must
come *after* these tags -->
+    <title>Apache Flink: Flink 2015: A year in review, and a lookout to 2016</title>
+    <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
+    <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+    <!-- Bootstrap -->
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css">
+    <link rel="stylesheet" href="/css/flink.css">
+    <link rel="stylesheet" href="/css/syntax.css">
+
+    <!-- Blog RSS feed -->
+    <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache
Flink Blog: RSS feed" />
+
+    <!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
+    <!-- We need to load Jquery in the header for custom google analytics event tracking-->
+    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></script>
+
+    <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries
-->
+    <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+    <!--[if lt IE 9]>
+      <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+      <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+    <![endif]-->
+  </head>
+  <body>  
+    
+
+  <!-- Top navbar. -->
+    <nav class="navbar navbar-default navbar-fixed-top">
+      <div class="container">
+        <!-- The logo. -->
+        <div class="navbar-header">
+          <button type="button" class="navbar-toggle collapsed" data-toggle="collapse"
data-target="#bs-example-navbar-collapse-1">
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+            <span class="icon-bar"></span>
+          </button>
+          <div class="navbar-logo">
+            <a href="/">
+              <img alt="Apache Flink" src="/img/navbar-brand-logo.jpg" width="78px" height="40px">
+            </a>
+          </div>
+        </div><!-- /.navbar-header -->
+
+        <!-- The navigation links. -->
+        <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+          <ul class="nav navbar-nav">
+
+            <!-- Overview -->
+            <li><a href="/index.html">Overview</a></li>
+
+            <!-- Features -->
+            <li><a href="/features.html">Features</a></li>
+
+            <!-- Downloads -->
+            <li><a href="/downloads.html">Downloads</a></li>
+
+            <!-- FAQ -->
+            <li><a href="/faq.html">FAQ</a></li>
+
+
+            <!-- Quickstart -->
+            <li class="dropdown">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button"
aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small>
Quickstart <span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-0.10/quickstart/setup_quickstart.html">Setup</a></li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-0.10/quickstart/java_api_quickstart.html">Java
API</a></li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-0.10/quickstart/scala_api_quickstart.html">Scala
API</a></li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-0.10/quickstart/run_example_quickstart.html">Run
Step-by-Step Example</a></li>
+              </ul>
+            </li>
+
+
+            <!-- Documentation -->
+            <li class="dropdown">
+              <a href="" class="dropdown-toggle" data-toggle="dropdown" role="button"
aria-expanded="false"><small><span class="glyphicon glyphicon-new-window"></span></small>
Documentation <span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <!-- Latest stable release -->
+                <li role="presentation" class="dropdown-header"><strong>Latest
Release</strong> (Stable)</li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-0.10">0.10.1
Documentation</a></li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-0.10/api/java"
class="active">0.10.1 Javadocs</a></li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-release-0.10/api/scala/index.html"
class="active">0.10.1 ScalaDocs</a></li>
+
+                <!-- Snapshot docs -->
+                <li class="divider"></li>
+                <li role="presentation" class="dropdown-header"><strong>Snapshot</strong>
(Development)</li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-master">1.0
Documentation</a></li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/java"
class="active">1.0 Javadocs</a></li>
+                <li><a href="http://ci.apache.org/projects/flink/flink-docs-master/api/scala/index.html"
class="active">1.0 ScalaDocs</a></li>
+
+                <!-- Wiki -->
+                <li class="divider"></li>
+                <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home">Wiki</a></li>
+              </ul>
+            </li>
+
+          </ul>
+
+          <ul class="nav navbar-nav navbar-right">
+            <!-- Blog -->
+            <li class=" active hidden-md hidden-sm"><a href="/blog/">Blog</a></li>
+
+            <li class="dropdown hidden-md hidden-sm">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button"
aria-expanded="false">Community <span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <!-- Community -->
+                <li role="presentation" class="dropdown-header"><strong>Community</strong></li>
+                <li><a href="/community.html#mailing-lists">Mailing Lists</a></li>
+                <li><a href="/community.html#irc">IRC</a></li>
+                <li><a href="/community.html#stack-overflow">Stack Overflow</a></li>
+                <li><a href="/community.html#issue-tracker">Issue Tracker</a></li>
+                <li><a href="/community.html#third-party-packages">Third Party
Packages</a></li>
+                <li><a href="/community.html#source-code">Source Code</a></li>
+                <li><a href="/community.html#people">People</a></li>
+                <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink"><small><span
class="glyphicon glyphicon-new-window"></span></small> Powered by Flink</a></li>
+
+                <!-- Contribute -->
+                <li class="divider"></li>
+                <li role="presentation" class="dropdown-header"><strong>Contribute</strong></li>
+                <li><a href="/how-to-contribute.html">How to Contribute</a></li>
+                <li><a href="/contribute-code.html">Contribute Code</a></li>
+                <li><a href="/contribute-documentation.html">Contribute Documentation</a></li>
+                <li><a href="/improve-website.html">Improve the Website</a></li>
+              </ul>
+            </li>
+
+            <li class="dropdown hidden-md hidden-sm">
+              <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button"
aria-expanded="false">Project <span class="caret"></span></a>
+              <ul class="dropdown-menu" role="menu">
+                <!-- Project -->
+                <li role="presentation" class="dropdown-header"><strong>Project</strong></li>
+                <li><a href="/material.html">Material</a></li>
+                <li><a href="https://twitter.com/apacheflink"><small><span
class="glyphicon glyphicon-new-window"></span></small> Twitter</a></li>
+                <li><a href="https://github.com/apache/flink"><small><span
class="glyphicon glyphicon-new-window"></span></small> GitHub</a></li>
+                <li><a href="https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home"><small><span
class="glyphicon glyphicon-new-window"></span></small> Wiki</a></li>
+              </ul>
+            </li>
+          </ul>
+        </div><!-- /.navbar-collapse -->
+      </div><!-- /.container -->
+    </nav>
+
+
+    <!-- Main content. -->
+    <div class="container">
+      
+
+<div class="row">
+  <div class="col-sm-8 col-sm-offset-2">
+    <div class="row">
+      <h1>Flink 2015: A year in review, and a lookout to 2016</h1>
+
+      <article>
+        <p>18 Dec 2015 by Robert Metzger (<a href="https://twitter.com/rmetzger_">@rmetzger_</a>)</p>
+
+<p>With 2015 ending, we thought that this would be good time to reflect
+on the amazing work done by the Flink community over this past year,
+and how much this community has grown.</p>
+
+<p>Overall, we have seen Flink grow in terms of functionality from an
+engine to one of the most complete open-source stream processing
+frameworks available. The community grew from a relatively small and
+geographically focused team, to a truly global, and one of the largest
+big data communities in the the Apache Software Foundation.</p>
+
+<p>We will also look at some interesting stats, including that the
+busiest days for Flink are Mondays (who would have thought :-).</p>
+
+<h1 id="community-growth">Community growth</h1>
+
+<p>Let us start with some simple statistics from <a href="https://github.com/apache/flink">Flink’s
+github repository</a>. During 2015, the
+Flink community <strong>doubled</strong> in size, from about 75 contributors
to
+over 150. Forks of the repository more than <strong>tripled</strong> from 160
in
+February 2015 to 544 in December 2015, and the number of stars of the
+repository almost tripled from 289 to 813.</p>
+
+<center>
+<img src="/img/blog/community-growth.png" style="height:400px;margin:15px" />
+</center>
+
+<p>Although Flink started out geographically in Berlin, Germany, the
+community is by now spread all around the globe, with many
+contributors from North America, Europe, and Asia. A simple search at
+meetup.com for groups that mention Flink as a focus area reveals <a href="http://apache-flink.meetup.com/">16
+meetups around the globe</a>:</p>
+
+<center>
+<img src="/img/blog/meetup-map.png" style="height:400px;margin:15px" />
+</center>
+
+<h1 id="flink-forward-2015">Flink Forward 2015</h1>
+
+<p>One of the highlights of the year for Flink was undoubtedly the <a href="http://flink-forward.org/">Flink
+Forward</a> conference, the first conference
+on Apache Flink that was held in October in Berlin. More than 250
+participants (roughly half based outside Germany where the conference
+was held) attended more than 33 technical talks from organizations
+including Google, MongoDB, Bouygues Telecom, NFLabs, Euranova, RedHat,
+IBM, Huawei, Intel, Ericsson, Capital One, Zalando, Amadeus, the Otto
+Group, and ResearchGate. If you have not yet watched their talks,
+check out the <a href="http://flink-forward.org/?post_type=day">slides</a> and
+<a href="https://www.youtube.com/playlist?list=PLDX4T_cnKjD31JeWR1aMOi9LXPRQ6nyHO">videos</a>
+from Flink Forward.</p>
+
+<center>
+<img src="/img/blog/ff-speakers.png" style="height:400px;margin:15px" />
+</center>
+
+<h1 id="media-coverage">Media coverage</h1>
+
+<p>And of course, interest in Flink was picked up by the tech
+media. During 2015, articles about Flink appeared in
+<a href="http://www.infoq.com/Apache-Flink/news/">InfoQ</a>,
+<a href="http://www.zdnet.com/article/five-open-source-big-data-projects-to-watch/">ZDNet</a>,
+<a href="http://www.datanami.com/tag/apache-flink/">Datanami</a>,
+<a href="http://www.infoworld.com/article/2919602/hadoop/flink-hadoops-new-contender-for-mapreduce-spark.html">Infoworld</a>
+(including being one of the <a href="http://www.infoworld.com/article/2982429/open-source-tools/bossie-awards-2015-the-best-open-source-big-data-tools.html">best
open source big data tools of
+2015</a>),
+the <a href="http://blogs.gartner.com/nick-heudecker/apache-flink-offers-a-challenge-to-spark/">Gartner
+blog</a>,
+<a href="http://dataconomy.com/tag/apache-flink/">Dataconomy</a>,
+<a href="http://sdtimes.com/tag/apache-flink/">SDTimes</a>, the <a href="https://www.mapr.com/blog/apache-flink-new-way-handle-streaming-data">MapR
+blog</a>,
+<a href="http://www.kdnuggets.com/2015/08/apache-flink-stream-processing.html">KDnuggets</a>,
+and
+<a href="http://www.hadoopsphere.com/2015/02/distributed-data-processing-with-apache.html">HadoopSphere</a>.</p>
+
+<center>
+<img src="/img/blog/appeared-in.png" style="height:400px;margin:15px" />
+</center>
+
+<p>It is interesting to see that Hadoop Summit EMEA 2016 had a whopping
+number of 17 (!) talks submitted that are mentioning Flink in their
+title and abstract:</p>
+
+<center>
+<img src="/img/blog/hadoop-summit.png" style="height:400px;margin:15px" />
+</center>
+
+<h1 id="fun-with-stats-when-do-committers-commit">Fun with stats: when do committers
commit?</h1>
+
+<p>To get some deeper insight on what is happening in the Flink
+community, let us do some analytics on the git log of the project :-)
+The easiest thing we can do is count the number of commits at the
+repository in 2015. Running</p>
+
+<div class="highlight"><pre><code>git log --pretty=oneline --after=1/1/2015
 | wc -l
+</code></pre></div>
+
+<p>on the Flink repository yields a total of <strong>2203 commits</strong>
in 2015.</p>
+
+<p>To dig deeper, we will use an open source tool called gitstats that
+will give us some interesting statistics on the committer
+behavior. You can create these also yourself and see many more by
+following four easy steps:</p>
+
+<ol>
+  <li>Download gitstats from the <a href="http://gitstats.sourceforge.net/">project
homepage</a>.. E.g., on OS X with homebrew, type</li>
+</ol>
+
+<div class="highlight"><pre><code>brew install --HEAD homebrew/head-only/gitstats
+</code></pre></div>
+
+<ol>
+  <li>Clone the Apache Flink git repository:</li>
+</ol>
+
+<div class="highlight"><pre><code>git clone git@github.com:apache/flink.git
+</code></pre></div>
+
+<ol>
+  <li>Generate the statistics</li>
+</ol>
+
+<div class="highlight"><pre><code>gitstats flink/ flink-stats/
+</code></pre></div>
+
+<ol>
+  <li>View all the statistics as an html page using your favorite browser (e.g., chrome):</li>
+</ol>
+
+<div class="highlight"><pre><code>chrome flink-stats/index.html
+</code></pre></div>
+
+<p>First, we can see a steady growth of lines of code in Flink since the
+initial Apache incubator project. During 2015, the codebase almost
+<strong>doubled</strong> from 500,000 LOC to 900,000 LOC.</p>
+
+<center>
+<img src="/img/blog/code-growth.png" style="height:400px;margin:15px" />
+</center>
+
+<p>It is interesting to see when committers commit. For Flink, Monday
+afternoons are by far the most popular times to commit to the
+repository:</p>
+
+<center>
+<img src="/img/blog/commit-stats.png" style="height:400px;margin:15px" />
+</center>
+
+<h1 id="feature-timeline">Feature timeline</h1>
+
+<p>So, what were the major features added to Flink and the Flink
+ecosystem during 2015? Here is a (non-exhaustive) chronological list:</p>
+
+<center>
+<img src="/img/blog/feature-timeline.png" style="height:400px;margin:15px" />
+</center>
+
+<h1 id="roadmap-for-2016">Roadmap for 2016</h1>
+
+<p>With 2015 coming to a close, the Flink community has already started
+discussing Flink’s roadmap for the future. Some highlights
+are:</p>
+
+<ul>
+  <li>
+    <p><strong>Runtime scaling of streaming jobs:</strong> streaming jobs
are running
+  forever, and need to react to a changing environment. Runtime
+  scaling means dynamically increasing and decreasing the
+  parallelism of a job to sustain certain SLAs, or react to changing
+  input throughput.</p>
+  </li>
+  <li>
+    <p><strong>SQL queries for static data sets and streams:</strong> building
on top of
+  Flink’s Table API, users should be able to write SQL
+  queries for static data sets, as well as SQL queries on data
+  streams that continuously produce new results.</p>
+  </li>
+  <li>
+    <p><strong>Streaming operators backed by managed memory:</strong> currently,
+  streaming operators like user-defined state and windows are backed
+  by JVM heap objects. Moving those to Flink managed memory will add
+  the ability to spill to disk, GC efficiency, as well as better
+  control over memory utilization.</p>
+  </li>
+  <li>
+    <p><strong>Library for detecting temporal event patterns:</strong>
a common use case
+  for stream processing is detecting patterns in an event stream
+  with timestamps. Flink makes this possible with its support for
+  event time, so many of these operators can be surfaced in the form
+  of a library.</p>
+  </li>
+  <li>
+    <p><strong>Support for Apache Mesos, and resource-dynamic YARN support:</strong>
+  support for both Mesos and YARN, including dynamic allocation and
+  release of resource for more resource elasticity (for both batch
+  and stream processing).</p>
+  </li>
+  <li>
+    <p><strong>Security:</strong> encrypt both the messages exchanged between
+  TaskManagers and JobManager, as well as the connections for data
+  exchange between workers.</p>
+  </li>
+  <li>
+    <p><strong>More streaming connectors, more runtime metrics, and continuous
+  DataStream API enhancements:</strong> add support for more sources and
+  sinks (e.g., Amazon Kinesis, Cassandra, Flume, etc), expose more
+  metrics to the user, and provide continuous improvements to the
+  DataStream API.</p>
+  </li>
+</ul>
+
+<p>If you are interested in these features, we highly encourage you to
+take a look at the <a href="https://docs.google.com/document/d/1ExmtVpeVVT3TIhO1JoBpC5JKXm-778DAD7eqw5GANwE/edit">current
+draft</a>,
+and <a href="https://mail-archives.apache.org/mod_mbox/flink-dev/201512.mbox/browser">join
the
+discussion</a>
+on the Flink mailing lists.</p>
+
+
+      </article>
+    </div>
+
+    <div class="row">
+      <div id="disqus_thread"></div>
+      <script type="text/javascript">
+        /* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */
+        var disqus_shortname = 'stratosphere-eu'; // required: replace example with your
forum shortname
+
+        /* * * DON'T EDIT BELOW THIS LINE * * */
+        (function() {
+            var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async
= true;
+            dsq.src = '//' + disqus_shortname + '.disqus.com/embed.js';
+             (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
+        })();
+      </script>
+    </div>
+  </div>
+</div>
+
+      <hr />
+      <div class="footer text-center">
+        <p>Copyright © 2014-2015 <a href="http://apache.org">The Apache Software
Foundation</a>. All Rights Reserved.</p>
+        <p>Apache Flink, Apache, and the Apache feather logo are trademarks of The
Apache Software Foundation.</p>
+        <p><a href="/privacy-policy.html">Privacy Policy</a> &middot;
<a href="/blog/feed.xml">RSS feed</a></p>
+      </div>
+
+    </div><!-- /.container -->
+
+    <!-- Include all compiled plugins (below), or include individual files as needed -->
+    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script>
+    <script src="/js/codetabs.js"></script>
+
+    <!-- Google Analytics -->
+    <script>
+      (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+      (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+      m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+      })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+      ga('create', 'UA-52545728-1', 'auto');
+      ga('send', 'pageview');
+    </script>
+  </body>
+</html>

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/img/blog/appeared-in.png
----------------------------------------------------------------------
diff --git a/img/blog/appeared-in.png b/img/blog/appeared-in.png
new file mode 100644
index 0000000..606da9e
Binary files /dev/null and b/img/blog/appeared-in.png differ

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/img/blog/code-growth.png
----------------------------------------------------------------------
diff --git a/img/blog/code-growth.png b/img/blog/code-growth.png
new file mode 100644
index 0000000..196c907
Binary files /dev/null and b/img/blog/code-growth.png differ

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/img/blog/commit-stats.png
----------------------------------------------------------------------
diff --git a/img/blog/commit-stats.png b/img/blog/commit-stats.png
new file mode 100644
index 0000000..dfeb7b3
Binary files /dev/null and b/img/blog/commit-stats.png differ

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/img/blog/community-growth.png
----------------------------------------------------------------------
diff --git a/img/blog/community-growth.png b/img/blog/community-growth.png
new file mode 100644
index 0000000..e37df51
Binary files /dev/null and b/img/blog/community-growth.png differ

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/img/blog/feature-timeline.png
----------------------------------------------------------------------
diff --git a/img/blog/feature-timeline.png b/img/blog/feature-timeline.png
new file mode 100644
index 0000000..1f56747
Binary files /dev/null and b/img/blog/feature-timeline.png differ

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/img/blog/ff-speakers.png
----------------------------------------------------------------------
diff --git a/img/blog/ff-speakers.png b/img/blog/ff-speakers.png
new file mode 100644
index 0000000..429a2c8
Binary files /dev/null and b/img/blog/ff-speakers.png differ

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/img/blog/hadoop-summit.png
----------------------------------------------------------------------
diff --git a/img/blog/hadoop-summit.png b/img/blog/hadoop-summit.png
new file mode 100644
index 0000000..3d9f594
Binary files /dev/null and b/img/blog/hadoop-summit.png differ

http://git-wip-us.apache.org/repos/asf/flink-web/blob/b90ef37c/img/blog/meetup-map.png
----------------------------------------------------------------------
diff --git a/img/blog/meetup-map.png b/img/blog/meetup-map.png
new file mode 100644
index 0000000..249dcf2
Binary files /dev/null and b/img/blog/meetup-map.png differ


Mime
View raw message