nemo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From won...@apache.org
Subject [incubator-nemo-website] branch asf-site updated: add blog post
Date Thu, 22 Mar 2018 16:21:34 GMT
This is an automated email from the ASF dual-hosted git repository.

wonook pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-nemo-website.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new 1a7d589  add blog post
1a7d589 is described below

commit 1a7d589d573335586d4c7c212595b913724adb37
Author: Won Wook SONG <wsong0512@gmail.com>
AuthorDate: Fri Mar 23 01:21:25 2018 +0900

    add blog post
---
 _posts/2018-03-20-nemo-blog-published.md           |   2 +-
 _posts/2018-03-23-shuffle-on-nemo.md               |  47 ++++++
 content/404.html                                   |   4 +-
 content/allposts.html                              |  11 +-
 .../blog/2018/03/20/nemo-blog-published/index.html |  16 +-
 content/blog/2018/03/23/shuffle-on-nemo/index.html | 181 +++++++++++++++++++++
 content/docs/bootstrap/index.html                  |   6 +-
 content/docs/cheatsheet/index.html                 |   6 +-
 content/docs/compiler_design/index.html            |   6 +-
 content/docs/extending_nemo/index.html             |   6 +-
 content/docs/font-awesome/index.html               |   6 +-
 content/docs/getting_started/index.html            |   6 +-
 content/docs/home/index.html                       |   6 +-
 content/docs/ir/index.html                         |   6 +-
 content/docs/jekyll/index.html                     |   6 +-
 content/docs/passes_and_policies/index.html        |   6 +-
 content/docs/runtime_design/index.html             |   6 +-
 content/docs/themes/index.html                     |   6 +-
 content/docs/welcome_to_jekyll/index.html          |   6 +-
 content/feed.xml                                   |  43 ++++-
 content/index.html                                 |   4 +-
 content/pages/downloads/index.html                 |   6 +-
 content/pages/license/index.html                   |   6 +-
 content/pages/talks/index.html                     |   6 +-
 content/pages/team/index.html                      |   6 +-
 content/search.json                                |   5 +
 content/sitemap.xml                                |  38 +++--
 27 files changed, 370 insertions(+), 83 deletions(-)

diff --git a/_posts/2018-03-20-nemo-blog-published.md b/_posts/2018-03-20-nemo-blog-published.md
index 1e501e0..bd5dff1 100644
--- a/_posts/2018-03-20-nemo-blog-published.md
+++ b/_posts/2018-03-20-nemo-blog-published.md
@@ -1,7 +1,7 @@
 ---
 layout: post
 title:  "Nemo blog published!"
-author: wonook
+author: Won Wook SONG
 ---
 Our blog is published and is online! We'll be posting exciting news related to our project on our blog.
 
diff --git a/_posts/2018-03-23-shuffle-on-nemo.md b/_posts/2018-03-23-shuffle-on-nemo.md
new file mode 100644
index 0000000..7b3c82c
--- /dev/null
+++ b/_posts/2018-03-23-shuffle-on-nemo.md
@@ -0,0 +1,47 @@
+---
+layout: post
+title:  "Optimizing shuffle performance using Nemo"
+author: John Yang
+---
+Data shuffle is a key operation that underlies almost all large-scale data processing jobs. A shuffle operation typically involves writing intermediate data to disk, and reading the data back later when the successive computations are scheduled.
+
+Sailfish[1] is an optimization technique that reduces disk overheads associated with a shuffle operation. Specifically, Sailfish minimizes the number of disk seeks involved in reading intermediate data back from disk. Jobs that handle large volumes of data can especially benefit from the Sailfish technique.
+
+Nemo provides an optimization policy interface that makes it easy for users to employ techniques like Sailfish to improve application performance. To demonstrate the flexibility of Nemo, we have developed and evaluated SailfishPolicy. We summarize preliminary evaluation results as follows.
+
+
+### Experimentation setup
+
+- Systems: Spark[2] 2.3.0 (a state-of-the-art system), and Nemo with SailfishPolicy
+- Resources: 20 h1.4xlarge (16 vCPU, 64GB memory, 2 HDDs) AWS instances 
+  - One of the disk is used by a HDFS cluster, and the other is used as a scratch disk by Nemo and Spark for maintaining intermediate data
+- Dataset: 2TB Wikipedia pageview statistics[3] stored in the HDFS cluster
+- Application: A MapReduce application that reads input data from HDFS, computes the sum of pageview counts per Wikipedia project, and writes the results to HDFS
+
+
+### Job completion time (JCT)
+
+![Figure 1](https://user-images.githubusercontent.com/6691311/37783061-d7c62970-2e37-11e8-89d5-9ef3da8fd846.png)
+<center>Figure 1</center>
+
+
+
+As shown in Figure 1, Nemo outperforms Spark by 2.26X primarily because Nemo’s reduce stage completes faster than Spark’s.
+
+
+### Mean disk throughput (MB/s)
+
+![Figure 2](https://user-images.githubusercontent.com/6691311/37783098-f17b55d4-2e37-11e8-9cf3-bf082562c1e6.png)
+<center>Figure 2</center>
+
+ 
+
+To understand the performance difference, we’ve measured the mean throughput of the scratch disks that Nemo and Spark use for handling intermediate data. As depicted in Figure 2, Nemo’s reduce stage enjoys much higher disk read throughput with a smaller number of disk seeks. This explains why Nemo’s reduce stage was able to complete more quickly, and validates the effectiveness of SailfishPolicy.
+
+
+
+[1] RAO, S., RAMAKRISHNAN, R., SILBERSTEIN, A., OVSIANNIKOV, M., AND REEVES, D. Sailfish: A framework for large scale data processing. In SOCC (2012).
+
+[2] Apache Spark. https://spark.apache.org/.
+
+[3] Wikipedia pageview statistics. https://dumps.wikimedia.org/other/pagecounts-raw/.
diff --git a/content/404.html b/content/404.html
index dfef109..32734ad 100644
--- a/content/404.html
+++ b/content/404.html
@@ -25,7 +25,7 @@
 <meta property="og:url" content="http://nemo.apache.org//404.html" />
 <meta property="og:site_name" content="Nemo" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"A Data Processing System for Flexible Employment With Different Deployment Characteristics.","name":null,"@type":"WebPage","headline":"Nemo","dateModified":null,"datePublished":null,"sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//404.html","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"A Data Processing System for Flexible Employment With Different Deployment Characteristics.","name":null,"@type":"WebPage","headline":"Nemo","dateModified":null,"datePublished":null,"url":"http://nemo.apache.org//404.html","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -57,7 +57,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/allposts.html b/content/allposts.html
index 7b8a11c..8fca445 100644
--- a/content/allposts.html
+++ b/content/allposts.html
@@ -25,7 +25,7 @@
 <meta property="og:url" content="http://nemo.apache.org//allposts.html" />
 <meta property="og:site_name" content="Nemo" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"A Data Processing System for Flexible Employment With Different Deployment Characteristics.","name":null,"@type":"WebPage","headline":"Blog","dateModified":null,"datePublished":null,"sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//allposts.html","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"A Data Processing System for Flexible Employment With Different Deployment Characteristics.","name":null,"@type":"WebPage","headline":"Blog","dateModified":null,"datePublished":null,"url":"http://nemo.apache.org//allposts.html","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -57,7 +57,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li  class="active" ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li  class="active" ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
@@ -81,8 +81,13 @@
     <div class="inner-content">
         <h1>All Posts</h1> 
         <p>
+            <strong><a href="/blog/2018/03/23/shuffle-on-nemo/">Optimizing shuffle performance using Nemo</a></strong>
+            <span>on March 23, 2018  by John Yang</span>
+        </p>
+        
+        <p>
             <strong><a href="/blog/2018/03/20/nemo-blog-published/">Nemo blog published!</a></strong>
-            <span>on March 20, 2018  by wonook</span>
+            <span>on March 20, 2018  by Won Wook SONG</span>
         </p>
         
     </div>
diff --git a/content/blog/2018/03/20/nemo-blog-published/index.html b/content/blog/2018/03/20/nemo-blog-published/index.html
index e4c619f..5550d93 100644
--- a/content/blog/2018/03/20/nemo-blog-published/index.html
+++ b/content/blog/2018/03/20/nemo-blog-published/index.html
@@ -17,7 +17,7 @@
     <!-- Begin Jekyll SEO tag v2.3.0 -->
 <title>Nemo blog published! | Nemo</title>
 <meta property="og:title" content="Nemo blog published!" />
-<meta name="author" content="wonook" />
+<meta name="author" content="Won Wook SONG" />
 <meta property="og:locale" content="en_US" />
 <meta name="description" content="Our blog is published and is online! We’ll be posting exciting news related to our project on our blog." />
 <meta property="og:description" content="Our blog is published and is online! We’ll be posting exciting news related to our project on our blog." />
@@ -27,7 +27,7 @@
 <meta property="og:type" content="article" />
 <meta property="article:published_time" content="2018-03-20T00:00:00+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":{"@type":"Person","name":"wonook"},"description":"Our blog is published and is online! We’ll be posting exciting news related to our project on our blog.","name":null,"@type":"BlogPosting","headline":"Nemo blog published!","dateModified":"2018-03-20T00:00:00+09:00","datePublished":"2018-03-20T00:00:00+09:00","sameAs":null,"image":null,"mainEntityOfPage":{"@type":"WebPage","@id":"http://nemo.apache.org//blog/2018/03/20/nemo-blog-published/"},"url":"http://nemo.a [...]
+{"publisher":null,"author":{"@type":"Person","name":"Won Wook SONG"},"description":"Our blog is published and is online! We’ll be posting exciting news related to our project on our blog.","name":null,"@type":"BlogPosting","headline":"Nemo blog published!","dateModified":"2018-03-20T00:00:00+09:00","datePublished":"2018-03-20T00:00:00+09:00","url":"http://nemo.apache.org//blog/2018/03/20/nemo-blog-published/","sameAs":null,"mainEntityOfPage":{"@type":"WebPage","@id":"http://nemo.apache.o [...]
 <!-- End Jekyll SEO tag -->
 
 
@@ -59,7 +59,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li  class="active" ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li  class="active" ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
@@ -87,6 +87,8 @@
                 <h4>RECENT POSTS</h4>
                 <ul class="list-unstyled post-list-container">
                     
+                    <li><a href="/blog/2018/03/23/shuffle-on-nemo/" >Optimizing shuffle performance using Nemo</a></li>
+                    
                     <li><a href="/blog/2018/03/20/nemo-blog-published/"  class="active" >Nemo blog published!</a></li>
                     
                     <li><a href="/allposts">All posts ...</a></li>
@@ -96,7 +98,7 @@
 
         <div class="col-md-8">
             <h1>Nemo blog published!</h1>
-            <p>Mar 20, 2018 • wonook</p>
+            <p>Mar 20, 2018 • Won Wook SONG</p>
             <div  id="markdown-content-container"><p>Our blog is published and is online! We’ll be posting exciting news related to our project on our blog.</p>
 
 <p>Your contribution is welcome!</p>
@@ -104,6 +106,12 @@
             <hr>
             <ul class="pager">
                  
+                <li class="next">
+                    <a href="/blog/2018/03/23/shuffle-on-nemo/">
+                      Newer <span aria-hidden="true">&rarr;</span>
+                    </a>
+                </li>
+                
             </ul>
         </div>
 
diff --git a/content/blog/2018/03/23/shuffle-on-nemo/index.html b/content/blog/2018/03/23/shuffle-on-nemo/index.html
new file mode 100644
index 0000000..2ad35e5
--- /dev/null
+++ b/content/blog/2018/03/23/shuffle-on-nemo/index.html
@@ -0,0 +1,181 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+    <meta charset="utf-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+
+    <title>Optimizing shuffle performance using Nemo</title>
+    <meta name="description" content="Data shuffle is a key operation that underlies almost all large-scale data processing jobs. A shuffle operation typically involves writing intermediate data ...">
+
+
+    <link rel="stylesheet" href="/css/main.css">
+    <link rel="stylesheet" href="/css/font-awesome.min.css">
+
+    <link rel="shortcut icon" href="/favicon.ico?1">
+    <!-- Begin Jekyll SEO tag v2.3.0 -->
+<title>Optimizing shuffle performance using Nemo | Nemo</title>
+<meta property="og:title" content="Optimizing shuffle performance using Nemo" />
+<meta name="author" content="John Yang" />
+<meta property="og:locale" content="en_US" />
+<meta name="description" content="Data shuffle is a key operation that underlies almost all large-scale data processing jobs. A shuffle operation typically involves writing intermediate data to disk, and reading the data back later when the successive computations are scheduled." />
+<meta property="og:description" content="Data shuffle is a key operation that underlies almost all large-scale data processing jobs. A shuffle operation typically involves writing intermediate data to disk, and reading the data back later when the successive computations are scheduled." />
+<link rel="canonical" href="http://nemo.apache.org//blog/2018/03/23/shuffle-on-nemo/" />
+<meta property="og:url" content="http://nemo.apache.org//blog/2018/03/23/shuffle-on-nemo/" />
+<meta property="og:site_name" content="Nemo" />
+<meta property="og:type" content="article" />
+<meta property="article:published_time" content="2018-03-23T00:00:00+09:00" />
+<script type="application/ld+json">
+{"publisher":null,"author":{"@type":"Person","name":"John Yang"},"description":"Data shuffle is a key operation that underlies almost all large-scale data processing jobs. A shuffle operation typically involves writing intermediate data to disk, and reading the data back later when the successive computations are scheduled.","name":null,"@type":"BlogPosting","headline":"Optimizing shuffle performance using Nemo","dateModified":"2018-03-23T00:00:00+09:00","datePublished":"2018-03-23T00:00 [...]
+<!-- End Jekyll SEO tag -->
+
+
+    <link rel="canonical" href="http://nemo.apache.org//blog/2018/03/23/shuffle-on-nemo/">
+    <link rel="alternate" type="application/rss+xml" title="Nemo" href="http://nemo.apache.org//feed.xml" />
+</head>
+
+
+<body>
+
+    <nav class="navbar navbar-default navbar-fixed-top">
+    <div class="container navbar-container">
+        <div class="navbar-header">
+            <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+        <span class="sr-only">Toggle navigation</span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+        <span class="icon-bar"></span>
+      </button>
+            <a class="navbar-brand" href="/">
+                <span><img src="/img/nemo-logo.png"></span>
+            </a>
+        </div>
+        <div id="navbar" class="collapse navbar-collapse">
+            <ul class="nav navbar-nav">
+                <li ><a href="/docs/home/">Docs</a></li>
+                <li ><a href="/apidocs">APIs</a></li>
+                <li ><a href="/pages/downloads">Downloads</a></li>
+                <li ><a href="/pages/talks">Talks</a></li>
+                <li ><a href="/pages/team">Team</a></li>
+                <li ><a href="/pages/license">License</a></li>
+                <li  class="active" ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
+            </ul>
+            <div class="navbar-right">
+                <form class="navbar-form navbar-left">
+                    <div class="form-group has-feedback">
+                        <input id="search-box" type="text" class="form-control" placeholder="Search...">
+                        <i class="fa fa-search form-control-feedback"></i>
+                    </div>
+                </form>
+                <ul class="nav navbar-nav">
+                    <li><a href="https://github.com/apache/incubator-nemo-website"><i class="fa fa-github" aria-hidden="true"></i></a></li>
+                </ul>
+            </div>
+        </div>
+    </div>
+</nav>
+
+
+    <div class="page-content">
+        <div class="wrapper">
+            <div class="container">
+    <div class="row">
+
+        <div class="col-md-4">
+            <div class="well">
+                <h4>RECENT POSTS</h4>
+                <ul class="list-unstyled post-list-container">
+                    
+                    <li><a href="/blog/2018/03/23/shuffle-on-nemo/"  class="active" >Optimizing shuffle performance using Nemo</a></li>
+                    
+                    <li><a href="/blog/2018/03/20/nemo-blog-published/" >Nemo blog published!</a></li>
+                    
+                    <li><a href="/allposts">All posts ...</a></li>
+                </ul>
+            </div>
+        </div>
+
+        <div class="col-md-8">
+            <h1>Optimizing shuffle performance using Nemo</h1>
+            <p>Mar 23, 2018 • John Yang</p>
+            <div  id="markdown-content-container"><p>Data shuffle is a key operation that underlies almost all large-scale data processing jobs. A shuffle operation typically involves writing intermediate data to disk, and reading the data back later when the successive computations are scheduled.</p>
+
+<p>Sailfish[1] is an optimization technique that reduces disk overheads associated with a shuffle operation. Specifically, Sailfish minimizes the number of disk seeks involved in reading intermediate data back from disk. Jobs that handle large volumes of data can especially benefit from the Sailfish technique.</p>
+
+<p>Nemo provides an optimization policy interface that makes it easy for users to employ techniques like Sailfish to improve application performance. To demonstrate the flexibility of Nemo, we have developed and evaluated SailfishPolicy. We summarize preliminary evaluation results as follows.</p>
+
+<h3 id="experimentation-setup">Experimentation setup</h3>
+
+<ul>
+  <li>Systems: Spark[2] 2.3.0 (a state-of-the-art system), and Nemo with SailfishPolicy</li>
+  <li>Resources: 20 h1.4xlarge (16 vCPU, 64GB memory, 2 HDDs) AWS instances
+    <ul>
+      <li>One of the disk is used by a HDFS cluster, and the other is used as a scratch disk by Nemo and Spark for maintaining intermediate data</li>
+    </ul>
+  </li>
+  <li>Dataset: 2TB Wikipedia pageview statistics[3] stored in the HDFS cluster</li>
+  <li>Application: A MapReduce application that reads input data from HDFS, computes the sum of pageview counts per Wikipedia project, and writes the results to HDFS</li>
+</ul>
+
+<h3 id="job-completion-time-jct">Job completion time (JCT)</h3>
+
+<p><img src="https://user-images.githubusercontent.com/6691311/37783061-d7c62970-2e37-11e8-89d5-9ef3da8fd846.png" alt="Figure 1" /></p>
+<center>Figure 1</center>
+
+<p>As shown in Figure 1, Nemo outperforms Spark by 2.26X primarily because Nemo’s reduce stage completes faster than Spark’s.</p>
+
+<h3 id="mean-disk-throughput-mbs">Mean disk throughput (MB/s)</h3>
+
+<p><img src="https://user-images.githubusercontent.com/6691311/37783098-f17b55d4-2e37-11e8-9cf3-bf082562c1e6.png" alt="Figure 2" /></p>
+<center>Figure 2</center>
+
+<p>To understand the performance difference, we’ve measured the mean throughput of the scratch disks that Nemo and Spark use for handling intermediate data. As depicted in Figure 2, Nemo’s reduce stage enjoys much higher disk read throughput with a smaller number of disk seeks. This explains why Nemo’s reduce stage was able to complete more quickly, and validates the effectiveness of SailfishPolicy.</p>
+
+<p>[1] RAO, S., RAMAKRISHNAN, R., SILBERSTEIN, A., OVSIANNIKOV, M., AND REEVES, D. Sailfish: A framework for large scale data processing. In SOCC (2012).</p>
+
+<p>[2] Apache Spark. https://spark.apache.org/.</p>
+
+<p>[3] Wikipedia pageview statistics. https://dumps.wikimedia.org/other/pagecounts-raw/.</p>
+</div>
+            <hr>
+            <ul class="pager">
+                
+                <li class="previous">
+                    <a href="/blog/2018/03/20/nemo-blog-published/">
+                        <span aria-hidden="true">&larr;</span> Older
+                    </a>
+                </li>
+                 
+            </ul>
+        </div>
+
+    </div>
+</div>
+
+        </div>
+    </div>
+
+    <footer class="footer">
+    <div class="container">
+
+        <p class="text-center">
+            Nemo 2018 |
+            Powered by <a href="https://github.com/aksakalli/jekyll-doc-theme">Jekyll Doc Theme</a>
+        </p>
+        <!-- <p class="text-muted">Place sticky footer content here.</p> -->
+    </div>
+</footer>
+
+    <script>
+  var baseurl = ''
+</script>
+<script src="//code.jquery.com/jquery-1.10.2.min.js"></script>
+<script src="/js/bootstrap.min.js "></script>
+<script src="/js/typeahead.bundle.min.js "></script>
+
+<script src="/js/main.js "></script>
+
+</body>
+
+</html>
diff --git a/content/docs/bootstrap/index.html b/content/docs/bootstrap/index.html
index 9b14f95..14d95fd 100644
--- a/content/docs/bootstrap/index.html
+++ b/content/docs/bootstrap/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/bootstrap/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Buttons Default Primary Success Info Warning Danger Link Default Primary Success Info Warning Danger Link Default Action Another action Something else here Separated link","name":null,"@type":"WebPage","headline":"Bootstrap Features","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//docs/bootstrap/","@context":"http://schema [...]
+{"publisher":null,"author":null,"description":"Buttons Default Primary Success Info Warning Danger Link Default Primary Success Info Warning Danger Link Default Action Another action Something else here Separated link","name":null,"@type":"WebPage","headline":"Bootstrap Features","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/bootstrap/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema [...]
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/cheatsheet/index.html b/content/docs/cheatsheet/index.html
index 7b894ca..0323321 100644
--- a/content/docs/cheatsheet/index.html
+++ b/content/docs/cheatsheet/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/cheatsheet/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"From adam-p/markdown-here","name":null,"@type":"WebPage","headline":"Markdown Cheatsheet","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//docs/cheatsheet/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"From adam-p/markdown-here","name":null,"@type":"WebPage","headline":"Markdown Cheatsheet","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/cheatsheet/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/compiler_design/index.html b/content/docs/compiler_design/index.html
index 958f692..2bfba00 100644
--- a/content/docs/compiler_design/index.html
+++ b/content/docs/compiler_design/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/compiler_design/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Overview","name":null,"@type":"WebPage","headline":"Compiler Design","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//docs/compiler_design/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"Overview","name":null,"@type":"WebPage","headline":"Compiler Design","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/compiler_design/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/extending_nemo/index.html b/content/docs/extending_nemo/index.html
index e8799b1..dc74b35 100644
--- a/content/docs/extending_nemo/index.html
+++ b/content/docs/extending_nemo/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/extending_nemo/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Overview","name":null,"@type":"WebPage","headline":"Extending Nemo","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//docs/extending_nemo/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"Overview","name":null,"@type":"WebPage","headline":"Extending Nemo","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/extending_nemo/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/font-awesome/index.html b/content/docs/font-awesome/index.html
index 6ce3248..e3c8787 100644
--- a/content/docs/font-awesome/index.html
+++ b/content/docs/font-awesome/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/font-awesome/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"500px address-book address-book-o address-card address-card-o adjust adn align-center align-justify align-left align-right amazon ambulance american-sign-language-interpreting anchor android angellist angle-double-down angle-double-left angle-double-right angle-double-up angle-down angle-left angle-right angle-up apple archive area-chart arrow-circle-down arrow-circle-left arrow-circle-o-down arrow-circle-o-left arrow-circle-o-right arrow-ci [...]
+{"publisher":null,"author":null,"description":"500px address-book address-book-o address-card address-card-o adjust adn align-center align-justify align-left align-right amazon ambulance american-sign-language-interpreting anchor android angellist angle-double-down angle-double-left angle-double-right angle-double-up angle-down angle-left angle-right angle-up apple archive area-chart arrow-circle-down arrow-circle-left arrow-circle-o-down arrow-circle-o-left arrow-circle-o-right arrow-ci [...]
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/getting_started/index.html b/content/docs/getting_started/index.html
index c54e9e8..bcd5f2a 100644
--- a/content/docs/getting_started/index.html
+++ b/content/docs/getting_started/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/getting_started/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Prerequisites Java 8 Maven Latest REEF snapshot YARN settings Download Hadoop 2.7.4 at http://apache.tt.co.kr/hadoop/common/hadoop-2.7.4/ Set the shell profile as following: export HADOOP_HOME=/path/to/hadoop-2.7.4 export YARN_HOME=$HADOOP_HOME export PATH=$PATH:$HADOOP_HOME/bin Protobuf 2.5.0 Downloadable at https://github.com/google/protobuf/releases/tag/v2.5.0 On Ubuntu: Run sudo apt-get install autoconf automake libtool curl make g++ unz [...]
+{"publisher":null,"author":null,"description":"Prerequisites Java 8 Maven Latest REEF snapshot YARN settings Download Hadoop 2.7.4 at http://apache.tt.co.kr/hadoop/common/hadoop-2.7.4/ Set the shell profile as following: export HADOOP_HOME=/path/to/hadoop-2.7.4 export YARN_HOME=$HADOOP_HOME export PATH=$PATH:$HADOOP_HOME/bin Protobuf 2.5.0 Downloadable at https://github.com/google/protobuf/releases/tag/v2.5.0 On Ubuntu: Run sudo apt-get install autoconf automake libtool curl make g++ unz [...]
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/home/index.html b/content/docs/home/index.html
index 49b2f39..3c2431d 100644
--- a/content/docs/home/index.html
+++ b/content/docs/home/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/home/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Nemo aims to optimize data processing for better performance and datacenter efficiency, not only in general and common conditions, but also with various deployment characteristics. Such characteristics include processing data on specific resource environments, like transient resources, and running jobs with specific attributes, like skewed data.","name":null,"@type":"WebPage","headline":"Overview","dateModified":"2018-03-20T16:53:35+09:00"," [...]
+{"publisher":null,"author":null,"description":"Nemo aims to optimize data processing for better performance and datacenter efficiency, not only in general and common conditions, but also with various deployment characteristics. Such characteristics include processing data on specific resource environments, like transient resources, and running jobs with specific attributes, like skewed data.","name":null,"@type":"WebPage","headline":"Overview","dateModified":"2018-03-23T01:20:55+09:00"," [...]
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/ir/index.html b/content/docs/ir/index.html
index 58e2d60..9bce862 100644
--- a/content/docs/ir/index.html
+++ b/content/docs/ir/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/ir/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Overview","name":null,"@type":"WebPage","headline":"Nemo Intermediate Representation (IR)","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//docs/ir/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"Overview","name":null,"@type":"WebPage","headline":"Nemo Intermediate Representation (IR)","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/ir/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/jekyll/index.html b/content/docs/jekyll/index.html
index 5836cac..b3487f6 100644
--- a/content/docs/jekyll/index.html
+++ b/content/docs/jekyll/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/jekyll/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Getting started","name":null,"@type":"WebPage","headline":"Jekyll intro","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//docs/jekyll/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"Getting started","name":null,"@type":"WebPage","headline":"Jekyll intro","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/jekyll/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/passes_and_policies/index.html b/content/docs/passes_and_policies/index.html
index 39df093..5db3f8e 100644
--- a/content/docs/passes_and_policies/index.html
+++ b/content/docs/passes_and_policies/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/passes_and_policies/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Optimization Passes","name":null,"@type":"WebPage","headline":"Passes and Policies","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//docs/passes_and_policies/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"Optimization Passes","name":null,"@type":"WebPage","headline":"Passes and Policies","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/passes_and_policies/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/runtime_design/index.html b/content/docs/runtime_design/index.html
index 3dc4166..59ebadc 100644
--- a/content/docs/runtime_design/index.html
+++ b/content/docs/runtime_design/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/runtime_design/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Receiving a Job from the Nemo Compiler","name":null,"@type":"WebPage","headline":"Runtime Design","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//docs/runtime_design/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"Receiving a Job from the Nemo Compiler","name":null,"@type":"WebPage","headline":"Runtime Design","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/runtime_design/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/themes/index.html b/content/docs/themes/index.html
index e7dd4ae..6e12a66 100644
--- a/content/docs/themes/index.html
+++ b/content/docs/themes/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/themes/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"By default, this website uses paper theme, but you can change it to another bootwatch theme by setting bootwatch variable in _config.yml file. (Don’t forget to restart Jekyll if you are running locally because the configuration is not re-read upon change.)","name":null,"@type":"WebPage","headline":"Themes","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null [...]
+{"publisher":null,"author":null,"description":"By default, this website uses paper theme, but you can change it to another bootwatch theme by setting bootwatch variable in _config.yml file. (Don’t forget to restart Jekyll if you are running locally because the configuration is not re-read upon change.)","name":null,"@type":"WebPage","headline":"Themes","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//docs/themes/","same [...]
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/docs/welcome_to_jekyll/index.html b/content/docs/welcome_to_jekyll/index.html
index 133f005..41b6808 100644
--- a/content/docs/welcome_to_jekyll/index.html
+++ b/content/docs/welcome_to_jekyll/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//docs/welcome_to_jekyll/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"You’ll find this post in your _posts directory. Go ahead and edit it and re-build the site to see your changes. You can rebuild the site in many different ways, but the most common way is to run jekyll serve, which launches a web server and auto-regenerates your site when a file is updated.","name":null,"@type":"WebPage","headline":"Welcome to Jekyll!","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sa [...]
+{"publisher":null,"author":null,"description":"You’ll find this post in your _posts directory. Go ahead and edit it and re-build the site to see your changes. You can rebuild the site in many different ways, but the most common way is to run jekyll serve, which launches a web server and auto-regenerates your site when a file is updated.","name":null,"@type":"WebPage","headline":"Welcome to Jekyll!","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","ur [...]
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/feed.xml b/content/feed.xml
index 0b05881..277cf01 100644
--- a/content/feed.xml
+++ b/content/feed.xml
@@ -1,4 +1,41 @@
-<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.4.3">Jekyll</generator><link href="http://nemo.apache.org//feed.xml" rel="self" type="application/atom+xml" /><link href="http://nemo.apache.org//" rel="alternate" type="text/html" /><updated>2018-03-20T16:53:35+09:00</updated><id>http://nemo.apache.org//</id><title type="html">Nemo</title><subtitle>A Data Processing System for Flexible Employment With Differ [...]
-</subtitle><entry><title type="html">Nemo blog published!</title><link href="http://nemo.apache.org//blog/2018/03/20/nemo-blog-published/" rel="alternate" type="text/html" title="Nemo blog published!" /><published>2018-03-20T00:00:00+09:00</published><updated>2018-03-20T00:00:00+09:00</updated><id>http://nemo.apache.org//blog/2018/03/20/nemo-blog-published</id><content type="html" xml:base="http://nemo.apache.org//blog/2018/03/20/nemo-blog-published/">&lt;p&gt;Our blog is published and i [...]
+<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.4.3">Jekyll</generator><link href="http://nemo.apache.org//feed.xml" rel="self" type="application/atom+xml" /><link href="http://nemo.apache.org//" rel="alternate" type="text/html" /><updated>2018-03-23T01:20:55+09:00</updated><id>http://nemo.apache.org//</id><title type="html">Nemo</title><subtitle>A Data Processing System for Flexible Employment With Differ [...]
+</subtitle><entry><title type="html">Optimizing shuffle performance using Nemo</title><link href="http://nemo.apache.org//blog/2018/03/23/shuffle-on-nemo/" rel="alternate" type="text/html" title="Optimizing shuffle performance using Nemo" /><published>2018-03-23T00:00:00+09:00</published><updated>2018-03-23T00:00:00+09:00</updated><id>http://nemo.apache.org//blog/2018/03/23/shuffle-on-nemo</id><content type="html" xml:base="http://nemo.apache.org//blog/2018/03/23/shuffle-on-nemo/">&lt;p& [...]
 
-&lt;p&gt;Your contribution is welcome!&lt;/p&gt;</content><author><name>wonook</name></author><summary type="html">Our blog is published and is online! We’ll be posting exciting news related to our project on our blog.</summary></entry></feed>
\ No newline at end of file
+&lt;p&gt;Sailfish[1] is an optimization technique that reduces disk overheads associated with a shuffle operation. Specifically, Sailfish minimizes the number of disk seeks involved in reading intermediate data back from disk. Jobs that handle large volumes of data can especially benefit from the Sailfish technique.&lt;/p&gt;
+
+&lt;p&gt;Nemo provides an optimization policy interface that makes it easy for users to employ techniques like Sailfish to improve application performance. To demonstrate the flexibility of Nemo, we have developed and evaluated SailfishPolicy. We summarize preliminary evaluation results as follows.&lt;/p&gt;
+
+&lt;h3 id=&quot;experimentation-setup&quot;&gt;Experimentation setup&lt;/h3&gt;
+
+&lt;ul&gt;
+  &lt;li&gt;Systems: Spark[2] 2.3.0 (a state-of-the-art system), and Nemo with SailfishPolicy&lt;/li&gt;
+  &lt;li&gt;Resources: 20 h1.4xlarge (16 vCPU, 64GB memory, 2 HDDs) AWS instances
+    &lt;ul&gt;
+      &lt;li&gt;One of the disk is used by a HDFS cluster, and the other is used as a scratch disk by Nemo and Spark for maintaining intermediate data&lt;/li&gt;
+    &lt;/ul&gt;
+  &lt;/li&gt;
+  &lt;li&gt;Dataset: 2TB Wikipedia pageview statistics[3] stored in the HDFS cluster&lt;/li&gt;
+  &lt;li&gt;Application: A MapReduce application that reads input data from HDFS, computes the sum of pageview counts per Wikipedia project, and writes the results to HDFS&lt;/li&gt;
+&lt;/ul&gt;
+
+&lt;h3 id=&quot;job-completion-time-jct&quot;&gt;Job completion time (JCT)&lt;/h3&gt;
+
+&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/6691311/37783061-d7c62970-2e37-11e8-89d5-9ef3da8fd846.png&quot; alt=&quot;Figure 1&quot; /&gt;&lt;/p&gt;
+&lt;center&gt;Figure 1&lt;/center&gt;
+
+&lt;p&gt;As shown in Figure 1, Nemo outperforms Spark by 2.26X primarily because Nemo’s reduce stage completes faster than Spark’s.&lt;/p&gt;
+
+&lt;h3 id=&quot;mean-disk-throughput-mbs&quot;&gt;Mean disk throughput (MB/s)&lt;/h3&gt;
+
+&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/6691311/37783098-f17b55d4-2e37-11e8-9cf3-bf082562c1e6.png&quot; alt=&quot;Figure 2&quot; /&gt;&lt;/p&gt;
+&lt;center&gt;Figure 2&lt;/center&gt;
+
+&lt;p&gt;To understand the performance difference, we’ve measured the mean throughput of the scratch disks that Nemo and Spark use for handling intermediate data. As depicted in Figure 2, Nemo’s reduce stage enjoys much higher disk read throughput with a smaller number of disk seeks. This explains why Nemo’s reduce stage was able to complete more quickly, and validates the effectiveness of SailfishPolicy.&lt;/p&gt;
+
+&lt;p&gt;[1] RAO, S., RAMAKRISHNAN, R., SILBERSTEIN, A., OVSIANNIKOV, M., AND REEVES, D. Sailfish: A framework for large scale data processing. In SOCC (2012).&lt;/p&gt;
+
+&lt;p&gt;[2] Apache Spark. https://spark.apache.org/.&lt;/p&gt;
+
+&lt;p&gt;[3] Wikipedia pageview statistics. https://dumps.wikimedia.org/other/pagecounts-raw/.&lt;/p&gt;</content><author><name>John Yang</name></author><summary type="html">Data shuffle is a key operation that underlies almost all large-scale data processing jobs. A shuffle operation typically involves writing intermediate data to disk, and reading the data back later when the successive computations are scheduled.</summary></entry><entry><title type="html">Nemo blog published!</title>< [...]
+
+&lt;p&gt;Your contribution is welcome!&lt;/p&gt;</content><author><name>Won Wook SONG</name></author><summary type="html">Our blog is published and is online! We’ll be posting exciting news related to our project on our blog.</summary></entry></feed>
\ No newline at end of file
diff --git a/content/index.html b/content/index.html
index 468950b..4b962ed 100644
--- a/content/index.html
+++ b/content/index.html
@@ -25,7 +25,7 @@
 <meta property="og:url" content="http://nemo.apache.org//" />
 <meta property="og:site_name" content="Nemo" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"A Data Processing System for Flexible Employment With Different Deployment Characteristics.","name":"Nemo","@type":"WebSite","headline":"Nemo","dateModified":null,"datePublished":null,"sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"A Data Processing System for Flexible Employment With Different Deployment Characteristics.","name":"Nemo","@type":"WebSite","headline":"Nemo","dateModified":null,"datePublished":null,"url":"http://nemo.apache.org//","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -57,7 +57,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/pages/downloads/index.html b/content/pages/downloads/index.html
index dba43af..4092f93 100644
--- a/content/pages/downloads/index.html
+++ b/content/pages/downloads/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//pages/downloads/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Releases","name":null,"@type":"WebPage","headline":"Downloads","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//pages/downloads/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"Releases","name":null,"@type":"WebPage","headline":"Downloads","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//pages/downloads/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/pages/license/index.html b/content/pages/license/index.html
index 5f7df88..d76b89b 100644
--- a/content/pages/license/index.html
+++ b/content/pages/license/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//pages/license/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"``` Apache License Version 2.0, January 2004 http://www.apache.org/licenses/","name":null,"@type":"WebPage","headline":"License","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//pages/license/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"``` Apache License Version 2.0, January 2004 http://www.apache.org/licenses/","name":null,"@type":"WebPage","headline":"License","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//pages/license/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li  class="active" ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/pages/talks/index.html b/content/pages/talks/index.html
index 9059bd5..f9b1512 100644
--- a/content/pages/talks/index.html
+++ b/content/pages/talks/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//pages/talks/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Onyx: A Flexible and Extensible Data Processing System, Deview 2017 (link) (Later renamed to Nemo)","name":null,"@type":"WebPage","headline":"Talks","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53:35+09:00","sameAs":null,"image":null,"mainEntityOfPage":null,"url":"http://nemo.apache.org//pages/talks/","@context":"http://schema.org"}</script>
+{"publisher":null,"author":null,"description":"Onyx: A Flexible and Extensible Data Processing System, Deview 2017 (link) (Later renamed to Nemo)","name":null,"@type":"WebPage","headline":"Talks","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20:55+09:00","url":"http://nemo.apache.org//pages/talks/","sameAs":null,"mainEntityOfPage":null,"image":null,"@context":"http://schema.org"}</script>
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li  class="active" ><a href="/pages/talks">Talks</a></li>
                 <li ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/pages/team/index.html b/content/pages/team/index.html
index 0be89b0..930665f 100644
--- a/content/pages/team/index.html
+++ b/content/pages/team/index.html
@@ -24,9 +24,9 @@
 <meta property="og:url" content="http://nemo.apache.org//pages/team/" />
 <meta property="og:site_name" content="Nemo" />
 <meta property="og:type" content="article" />
-<meta property="article:published_time" content="2018-03-20T16:53:35+09:00" />
+<meta property="article:published_time" content="2018-03-23T01:20:55+09:00" />
 <script type="application/ld+json">
-{"publisher":null,"author":null,"description":"Contributing to Nemo Your contributation is always welcome :) If you would like to contribute to Nemo, please subscribe to our mailing list: dev@nemo.apache.org [subscribe]. We have a Slack channel for active discussions. Join us on Slack by requesting for an invitation by sending an e-mail to dev@ after subscription.","name":null,"@type":"WebPage","headline":"Team","dateModified":"2018-03-20T16:53:35+09:00","datePublished":"2018-03-20T16:53 [...]
+{"publisher":null,"author":null,"description":"Contributing to Nemo Your contributation is always welcome :) If you would like to contribute to Nemo, please subscribe to our mailing list: dev@nemo.apache.org [subscribe]. We have a Slack channel for active discussions. Join us on Slack by requesting for an invitation by sending an e-mail to dev@ after subscription.","name":null,"@type":"WebPage","headline":"Team","dateModified":"2018-03-23T01:20:55+09:00","datePublished":"2018-03-23T01:20 [...]
 <!-- End Jekyll SEO tag -->
 
 
@@ -58,7 +58,7 @@
                 <li ><a href="/pages/talks">Talks</a></li>
                 <li  class="active" ><a href="/pages/team">Team</a></li>
                 <li ><a href="/pages/license">License</a></li>
-                <li ><a href="/blog/2018/03/20/nemo-blog-published/">Blog</a></li>
+                <li ><a href="/blog/2018/03/23/shuffle-on-nemo/">Blog</a></li>
             </ul>
             <div class="navbar-right">
                 <form class="navbar-form navbar-left">
diff --git a/content/search.json b/content/search.json
index 0bd78e7..1cae7b2 100644
--- a/content/search.json
+++ b/content/search.json
@@ -57,6 +57,11 @@
 
   
   {
+  "title": "Optimizing shuffle performance using Nemo",
+  "url": "/blog/2018/03/23/shuffle-on-nemo/"
+  },
+  
+  {
   "title": "Nemo blog published!",
   "url": "/blog/2018/03/20/nemo-blog-published/"
   }
diff --git a/content/sitemap.xml b/content/sitemap.xml
index 3b92735..969ed35 100644
--- a/content/sitemap.xml
+++ b/content/sitemap.xml
@@ -2,77 +2,81 @@
 <urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
 <url>
 <loc>http://nemo.apache.org//docs/compiler_design/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/runtime_design/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/getting_started/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/home/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/bootstrap/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/cheatsheet/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/jekyll/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/font-awesome/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/themes/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/welcome_to_jekyll/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/extending_nemo/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/ir/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//docs/passes_and_policies/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//pages/downloads/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//pages/license/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//pages/talks/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//pages/team/</loc>
-<lastmod>2018-03-20T16:53:35+09:00</lastmod>
+<lastmod>2018-03-23T01:20:55+09:00</lastmod>
 </url>
 <url>
 <loc>http://nemo.apache.org//blog/2018/03/20/nemo-blog-published/</loc>
 <lastmod>2018-03-20T00:00:00+09:00</lastmod>
 </url>
 <url>
+<loc>http://nemo.apache.org//blog/2018/03/23/shuffle-on-nemo/</loc>
+<lastmod>2018-03-23T00:00:00+09:00</lastmod>
+</url>
+<url>
 <loc>http://nemo.apache.org//allposts.html</loc>
 </url>
 <url>

-- 
To stop receiving notification emails like this one, please contact
wonook@apache.org.

Mime
View raw message