flink-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m..@apache.org
Subject [54/54] flink-web git commit: regenerate site
Date Wed, 01 Jul 2015 10:01:46 GMT
regenerate site


Project: http://git-wip-us.apache.org/repos/asf/flink-web/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink-web/commit/094bb9b3
Tree: http://git-wip-us.apache.org/repos/asf/flink-web/tree/094bb9b3
Diff: http://git-wip-us.apache.org/repos/asf/flink-web/diff/094bb9b3

Branch: refs/heads/asf-site
Commit: 094bb9b3e64275d19849b800edf5df3f9af7f717
Parents: bf8c853
Author: Maximilian Michels <mxm@apache.org>
Authored: Wed Jul 1 11:59:03 2015 +0200
Committer: Maximilian Michels <mxm@apache.org>
Committed: Wed Jul 1 11:59:03 2015 +0200

----------------------------------------------------------------------
 content/faq.html | 51 ++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 48 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flink-web/blob/094bb9b3/content/faq.html
----------------------------------------------------------------------
diff --git a/content/faq.html b/content/faq.html
index 1eb3f37..b1e7c3d 100644
--- a/content/faq.html
+++ b/content/faq.html
@@ -5,7 +5,7 @@
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=1">
     <!-- The above 3 meta tags *must* come first in the head; any other head content must
come *after* these tags -->
-    <title>Apache Flink: F.A.Q.</title>
+    <title>Apache Flink: Frequently Asked Questions (FAQ)</title>
     <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
     <link rel="icon" href="/favicon.ico" type="image/x-icon">
 
@@ -140,9 +140,28 @@
 <div class="row">
   <div class="col-sm-8 col-sm-offset-2">
     <div class="row">
-      <div class="col-sm-12"><h1>F.A.Q.</h1></div>
+      <div class="col-sm-12"><h1>Frequently Asked Questions (FAQ)</h1></div>
     </div>
 
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
 <p>The following questions are frequently asked with regard to the Flink project <strong>in
general</strong>. If you have further questions, make sure to consult the <a href="">documentation</a>
or <a href="">ask the community</a>.</p>
 
 <div class="page-toc">
@@ -164,6 +183,7 @@
       <li><a href="#in-scala-api-i-get-an-error-about-implicit-values-and-evidence-parameters">In
Scala API, I get an error about implicit values and evidence parameters</a></li>
       <li><a href="#i-get-an-error-message-saying-that-not-enough-buffers-are-available-how-do-i-fix-this">I
get an error message saying that not enough buffers are available. How do I fix this?</a></li>
       <li><a href="#my-job-fails-early-with-a-javaioeofexception-what-could-be-the-cause">My
job fails early with a java.io.EOFException. What could be the cause?</a></li>
+      <li><a href="#my-job-fails-with-various-exceptions-from-the-hdfshadoop-code-what-can-i-do">My
job fails with various exceptions from the HDFS/Hadoop code. What can I do?</a></li>
       <li><a href="#in-eclipse-i-get-compilation-errors-in-the-scala-projects">In
Eclipse, I get compilation errors in the Scala projects</a></li>
       <li><a href="#my-program-does-not-compute-the-correct-result-why-are-my-custom-key-types">My
program does not compute the correct result. Why are my custom key types</a></li>
       <li><a href="#i-get-a-javalanginstantiationexception-for-my-data-type-what-is-wrong">I
get a java.lang.InstantiationException for my data type, what is wrong?</a></li>
@@ -273,7 +293,7 @@ parallelism has to be 1 and set it accordingly.</p>
 <p>The parallelism can be set in numerous ways to ensure a fine-grained control
 over the execution of a Flink program. See
 the <a href="http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#common-options">Configuration
guide</a> for detailed instructions on how to
-set the parallelism. Also check out <a href="http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#configuring-taskmanager-processing-slots">this
figure</a> detailing 
+set the parallelism. Also check out <a href="http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#configuring-taskmanager-processing-slots">this
figure</a> detailing
 how the processing slots and parallelism are related to each other.</p>
 
 <h2 id="errors">Errors</h2>
@@ -343,6 +363,31 @@ breaks.</p>
 the https://github.com/apache/flink/tree/master/README.md
 for details on how to set up Flink for different Hadoop and HDFS versions.</p>
 
+<h3 id="my-job-fails-with-various-exceptions-from-the-hdfshadoop-code-what-can-i-do">My
job fails with various exceptions from the HDFS/Hadoop code. What can I do?</h3>
+
+<p>Flink is shipping with the Hadoop 2.2 binaries by default. These binaries are used
+to connect to HDFS or YARN.
+It seems that there are some bugs in the HDFS client which cause exceptions while writing
to HDFS
+(in particular under high load).
+Among the exceptions are the following:</p>
+
+<ul>
+  <li><code>HDFS client trying to connect to the standby Namenode "org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException):
Operation category READ is not supported in state standby"</code></li>
+  <li>
+    <p><code>java.io.IOException: Bad response ERROR for block BP-1335380477-172.22.5.37-1424696786673:blk_1107843111_34301064
from datanode 172.22.5.81:50010
+at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:732)</code></p>
+  </li>
+  <li><code>Caused by: org.apache.hadoop.ipc.RemoteException(java.lang.ArrayIndexOutOfBoundsException):
0
+      at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.getDatanodeStorageInfos(DatanodeManager.java:478)
+      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.updatePipelineInternal(FSNamesystem.java:6039)
+      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.updatePipeline(FSNamesystem.java:6002)</code></li>
+</ul>
+
+<p>If you are experiencing any of these, we recommend using a Flink build with a Hadoop
version matching
+your local HDFS version.
+You can also manually build Flink against the exact Hadoop version (for example
+when using a Hadoop distribution with a custom patch level)</p>
+
 <h3 id="in-eclipse-i-get-compilation-errors-in-the-scala-projects">In Eclipse, I get
compilation errors in the Scala projects</h3>
 
 <p>Flink uses a new feature of the Scala compiler (called “quasiquotes”) that have
not yet been properly


Mime
View raw message