kylin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lid...@apache.org
Subject svn commit: r1766917 - in /kylin/site: docs15/gettingstarted/faq.html feed.xml
Date Fri, 28 Oct 2016 02:28:19 GMT
Author: lidong
Date: Fri Oct 28 02:28:18 2016
New Revision: 1766917

URL: http://svn.apache.org/viewvc?rev=1766917&view=rev
Log:
update faq for EMR

Modified:
    kylin/site/docs15/gettingstarted/faq.html
    kylin/site/feed.xml

Modified: kylin/site/docs15/gettingstarted/faq.html
URL: http://svn.apache.org/viewvc/kylin/site/docs15/gettingstarted/faq.html?rev=1766917&r1=1766916&r2=1766917&view=diff
==============================================================================
--- kylin/site/docs15/gettingstarted/faq.html (original)
+++ kylin/site/docs15/gettingstarted/faq.html Fri Oct 28 02:28:18 2016
@@ -2104,26 +2104,52 @@
 							<h4 id="binfind-hive-dependencysh-can-locate-hivehcat-jars-in-local-but-kylin-reports-error-like-javalangnoclassdeffounderror-orgapachehivehcatalogmapreducehcatinputformat">1.
“bin/find-hive-dependency.sh” can locate hive/hcat jars in local, but Kylin reports
error like “java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/mapreduce/HCatInputFormat”</h4>
 
 <ul>
-  <li>
-    <p>Kylin need many dependent jars (hadoop/hive/hcat/hbase/kafka) on classpath to
work, but Kylin doesn’t ship them. It will seek these jars from your local machine by
running commands like <code class="highlighter-rouge">hbase classpath</code>,
<code class="highlighter-rouge">hive -e set</code> etc. The founded jars’
path will be appended to the environment variable <em>HBASE_CLASSPATH</em> (Kylin
uses <code class="highlighter-rouge">hbase</code> shell command to start up, which
will read this). But in some Hadoop distribution (like EMR 5.0), the <code class="highlighter-rouge">hbase</code>
shell doesn’t keep the origin <code class="highlighter-rouge">HBASE_CLASSPATH</code>
value, that causes the “NoClassDefFoundError”.</p>
-  </li>
-  <li>
-    <p>To fix this, find the hbase shell script (in hbase/bin folder), and search <em>HBASE_CLASSPATH</em>,
check whether it overwrite the value like :</p>
-  </li>
+  <li>Kylin need many dependent jars (hadoop/hive/hcat/hbase/kafka) on classpath to
work, but Kylin doesn’t ship them. It will seek these jars from your local machine by
running commands like <code class="highlighter-rouge">hbase classpath</code>,
<code class="highlighter-rouge">hive -e set</code> etc. The founded jars’
path will be appended to the environment variable <em>HBASE_CLASSPATH</em> (Kylin
uses <code class="highlighter-rouge">hbase</code> shell command to start up, which
will read this). But in some Hadoop distribution (like EMR 5.0), the <code class="highlighter-rouge">hbase</code>
shell doesn’t keep the origin <code class="highlighter-rouge">HBASE_CLASSPATH</code>
value, that causes the “NoClassDefFoundError”.</li>
 </ul>
 
-<div class="highlight"><pre><code class="language-groff" data-lang="groff">export
HBASE_CLASSPATH=$HADOOP_CONF:$HADOOP_HOME/*:$HADOOP_HOME/lib/*:$ZOOKEEPER_HOME/*:$ZOOKEEPER_HOME/lib/*</code></pre></div>
+<p>To fix this, find the hbase shell script (in hbase/bin or hbase/conf folder), and
search <em>HBASE_CLASSPATH</em>, check whether it overwrite the value like :</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>  export HBASE_CLASSPATH=$HADOOP_CONF:$HADOOP_HOME/*:$HADOOP_HOME/lib/*:$ZOOKEEPER_HOME/*:$ZOOKEEPER_HOME/lib/*
+</code></pre>
+</div>
+
+<p>If true, change it to keep the origin value like:</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>  export HBASE_CLASSPATH=$HADOOP_CONF:$HADOOP_HOME/*:$HADOOP_HOME/lib/*:$ZOOKEEPER_HOME/*:$ZOOKEEPER_HOME/lib/*:$HBASE_CLASSPATH
+</code></pre>
+</div>
 
 <ul>
-  <li>If true, change it to keep the origin value like:</li>
+  <li>For EMR5.0, need change the hbase-env.sh</li>
 </ul>
 
-<div class="highlight"><pre><code class="language-groff" data-lang="groff">export
HBASE_CLASSPATH=$HADOOP_CONF:$HADOOP_HOME/*:$HADOOP_HOME/lib/*:$ZOOKEEPER_HOME/*:$ZOOKEEPER_HOME/lib/*:$HBASE_CLASSPATH</code></pre></div>
+<div class="highlighter-rouge"><pre class="highlight"><code> sudo vi /usr/lib/hbase/conf/hbase-env.sh
+</code></pre>
+</div>
+
+<p>in around line 30 it was :</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>export HBASE_CLASSPATH=/etc/hadoop/conf
+</code></pre>
+</div>
+
+<p>change to :</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>export HBASE_CLASSPATH=/etc/hadoop/conf:$HBASE_CLASSPATH
+</code></pre>
+</div>
+
+<p>then restart Kylin.</p>
 
 <h4 id="get-javalangillegalargumentexception-too-high-cardinality-is-not-suitable-for-dictionary----cardinality-5220674-in-build-dimension-dictionary-step">2.
Get “java.lang.IllegalArgumentException: Too high cardinality is not suitable for dictionary
– cardinality: 5220674” in “Build Dimension Dictionary” step</h4>
 
 <ul>
-  <li>Kylin uses “Dictionary” encoding to encode/decode the dimension values
(check <a href="/blog/2015/08/13/kylin-dictionary/">this blog</a>); Usually a
dimension’s cardinality is less than millions, so the “Dict” encoding is good
to use. As dictionary need be persisted and loaded into memory, if a dimension’s cardinality
is very high, the memory footprint will be tremendous, so Kylin add a check on this. If you
see this error, suggest to identify the UHC dimension first and then re-evaluate the design
(whether need to make that as dimension?). If must keep it, you can by-pass this error with
couple ways: 1) change to use other encoding (like <code class="highlighter-rouge">fixed_length</code>,
<code class="highlighter-rouge">integer</code>) 2) or set a bigger value for <code
class="highlighter-rouge">kylin.dictionary.max.cardinality</code> in <code class="highlighter-rouge">conf/kylin.properties</code>.</li>
+  <li>Kylin uses “Dictionary” encoding to encode/decode the dimension values
(check <a href="/blog/2015/08/13/kylin-dictionary/">this blog</a>); Usually a
dimension’s cardinality is less than millions, so the “Dict” encoding is good
to use. As dictionary need be persisted and loaded into memory, if a dimension’s cardinality
is very high, the memory footprint will be tremendous, so Kylin add a check on this. If you
see this error, suggest to identify the UHC dimension first and then re-evaluate the design
(whether need to make that as dimension?). If must keep it, you can by-pass this error with
couple ways:
+    <ul>
+      <li>change to use other encoding (like <code class="highlighter-rouge">fixed_length</code>,
<code class="highlighter-rouge">integer</code>)</li>
+      <li>or set a bigger value for <code class="highlighter-rouge">kylin.dictionary.max.cardinality</code>
in <code class="highlighter-rouge">conf/kylin.properties</code>.</li>
+    </ul>
+  </li>
 </ul>
 
 <h4 id="build-cube-failed-due-to-error-check-status">3. Build cube failed due to “error
check status”</h4>

Modified: kylin/site/feed.xml
URL: http://svn.apache.org/viewvc/kylin/site/feed.xml?rev=1766917&r1=1766916&r2=1766917&view=diff
==============================================================================
--- kylin/site/feed.xml (original)
+++ kylin/site/feed.xml Fri Oct 28 02:28:18 2016
@@ -19,8 +19,8 @@
     <description>Apache Kylin Home</description>
     <link>http://kylin.apache.org/</link>
     <atom:link href="http://kylin.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
-    <pubDate>Wed, 26 Oct 2016 08:06:28 -0700</pubDate>
-    <lastBuildDate>Wed, 26 Oct 2016 08:06:28 -0700</lastBuildDate>
+    <pubDate>Fri, 28 Oct 2016 03:24:28 -0700</pubDate>
+    <lastBuildDate>Fri, 28 Oct 2016 03:24:28 -0700</lastBuildDate>
     <generator>Jekyll v2.5.3</generator>
     
       <item>



Mime
View raw message