tajo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hyun...@apache.org
Subject git commit: TAJO-669: Add cluster setup documentation. (hyunsik)
Date Tue, 11 Mar 2014 17:39:21 GMT
Repository: incubator-tajo
Updated Branches:
  refs/heads/branch-0.8.0 113574071 -> 1069efb3c


TAJO-669: Add cluster setup documentation. (hyunsik)


Project: http://git-wip-us.apache.org/repos/asf/incubator-tajo/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-tajo/commit/1069efb3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-tajo/tree/1069efb3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-tajo/diff/1069efb3

Branch: refs/heads/branch-0.8.0
Commit: 1069efb3ce5e24a7a29bdbcce94d565218266c17
Parents: 1135740
Author: Hyunsik Choi <hyunsik@apache.org>
Authored: Wed Mar 12 02:08:48 2014 +0900
Committer: Hyunsik Choi <hyunsik@apache.org>
Committed: Wed Mar 12 02:39:11 2014 +0900

----------------------------------------------------------------------
 CHANGES.txt                                     |  2 +
 tajo-docs/src/main/sphinx/cli.rst               |  2 +-
 .../main/sphinx/configuration/cluster_setup.rst | 64 ++++++++++++++++++--
 .../main/sphinx/getting_started/building.rst    |  2 +-
 .../main/sphinx/getting_started/first_query.rst | 26 +++++---
 .../main/sphinx/getting_started/local_setup.rst | 17 +++---
 6 files changed, 92 insertions(+), 21 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/1069efb3/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index 398c0d9..e56ed76 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -524,6 +524,8 @@ Release 0.8.0 - unreleased
 
   TASKS
 
+    TAJO-669: Add cluster setup documentation. (hyunsik)
+
     TAJO-681: Embed sphinx rtd theme into tajo-docs. (hyunsik)
 
     TAJO-657: Missing table stat in RCFile. (jinho)

http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/1069efb3/tajo-docs/src/main/sphinx/cli.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/cli.rst b/tajo-docs/src/main/sphinx/cli.rst
index f2fe60c..6c91d2f 100644
--- a/tajo-docs/src/main/sphinx/cli.rst
+++ b/tajo-docs/src/main/sphinx/cli.rst
@@ -1,5 +1,5 @@
 *****************************
-Command Line Interface (TSQL)
+Tajo Shell (TSQL)
 *****************************
 
 ==========

http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/1069efb3/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst b/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst
index d566ecb..e9715f3 100644
--- a/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst
+++ b/tajo-docs/src/main/sphinx/configuration/cluster_setup.rst
@@ -1,10 +1,66 @@
 *******************************************
-Tajo run modes: Standalone and Distributed
+Custer Setup
 *******************************************
 
+Fully Distributed Mode
+==========================================
+A fully distributed mode enables a Tajo instance to run on `Hadoop Distributed File System
(HDFS) <http://wiki.apache.org/hadoop/HDFS>`_. In this mode, a number of Tajo workers
run across a number of the physical nodes where HDFS data nodes run.
 
 
-==========================================
-Standalone Mode
-==========================================
+In this section, we explain how to setup the cluster mode. 
+
+
+Settings
+--------------------------------------------------------
+
+Please add the following configs to tajo-site.xml file:
+
+.. code-block:: xml
+
+  <property>
+    <name>tajo.rootdir</name>
+    <value>hdfs://hostname:port/tajo</value>
+  </property>
+
+  <property>
+    <name>tajo.master.umbilical-rpc.address</name>
+    <value>hostname:26001</value>
+  </property>
+
+  <property>
+    <name>tajo.master.client-rpc.addres</name>
+    <value>hostname:26002</value>
+  </property>
+
+  <property>
+    <name>tajo.catalog.client-rpc.address</name>
+    <value>hostname:26005</value>
+  </property>
+
+
+Make base directories and set permissions
+--------------------------------------------------------
+
+If you want to know Tajo’s configuration in more detail, see Configuration page.
+Before launching the tajo, you should create the tajo root dir and set the permission as
follows: ::
+
+  $ $HADOOP_HOME/bin/hadoop fs -mkdir       /tajo
+  $ $HADOOP_HOME/bin/hadoop fs -chmod g+w   /tajo
+
+
+Launch a Tajo cluster
+--------------------------------------------------------
+
+Then, execute ``start-tajo.sh`` ::
+
+  $ $TAJO_HOME/bin/start-tajo.sh
+
+.. note::
+
+  In default, each worker is set to very little resource capacity. In order to increase parallel
degree, please read 
+  :doc:`/configuration/worker_configuration`.
+
+.. note::
+
+  In default, TajoMaster listens on 127.0.0.1 for clients. To allow remote clients to access
TajoMaster, please set tajo.master.client-rpc.address config to tajo-site.xml. In order to
know how to change the listen port, please refer :doc:`/configuration/configuration_defaults`.
 

http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/1069efb3/tajo-docs/src/main/sphinx/getting_started/building.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/getting_started/building.rst b/tajo-docs/src/main/sphinx/getting_started/building.rst
index b5a7381..fbe003b 100644
--- a/tajo-docs/src/main/sphinx/getting_started/building.rst
+++ b/tajo-docs/src/main/sphinx/getting_started/building.rst
@@ -1,5 +1,5 @@
 *****************
-Build Source Code
+Build source code
 *****************
 
 You prepare the prerequisites and the source code, you can build the source code now.

http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/1069efb3/tajo-docs/src/main/sphinx/getting_started/first_query.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/getting_started/first_query.rst b/tajo-docs/src/main/sphinx/getting_started/first_query.rst
index fd43619..aa5f4d2 100644
--- a/tajo-docs/src/main/sphinx/getting_started/first_query.rst
+++ b/tajo-docs/src/main/sphinx/getting_started/first_query.rst
@@ -2,7 +2,9 @@
 First query execution
 ************************
 
-First of all, we need to prepare some data for query execution. For example, you can make
a simple text-based table as follows: ::
+First of all, we need to prepare some table for query execution. For example, you can make
a simple text-based table as follows: 
+
+.. code-block:: bash
 
   $ mkdir /home/x/table1
   $ cd /home/x/table1
@@ -14,7 +16,14 @@ First of all, we need to prepare some data for query execution. For example,
you
   5|mno|5.6|e
   <CTRL + D>
 
-This schema of this table is (int, text, float, text). ::
+
+Apache Tajo™ provides a SQL shell which allows users to interactively submit SQL queries.
In order to use this shell, please execute ``bin/tsql`` ::
+
+  $ $TAJO_HOME/bin/tsql
+  tajo>
+
+In order to load the table we created above, we should think of a schema of the table.
+Here, we assume the schema as (int, text, float, text). ::
 
   $ $TAJO_HOME/bin/tsql
   tajo> create external table table1 (
@@ -24,7 +33,7 @@ This schema of this table is (int, text, float, text). ::
         type text) 
         using csv with ('csvfile.delimiter'='|') location 'file:/home/x/table1';
 
-In order to load an external table, you need to use ‘create external table’ statement.

+To load an external table, you need to use ‘create external table’ statement. 
 In the location clause, you should use the absolute directory path with an appropriate scheme.

 If the table resides in HDFS, you should use ‘hdfs’ instead of ‘file’.
 
@@ -33,7 +42,7 @@ If you want to know DDL statements in more detail, please see Query Language.
::
   tajo> \d
   table1
 
-‘d’ command shows the list of tables. ::
+ ``\d`` command shows the list of tables. ::
 
   tajo> \d table1
 
@@ -48,7 +57,7 @@ If you want to know DDL statements in more detail, please see Query Language.
::
   score   FLOAT
   type    TEXT
 
-‘d [table name]’ command shows the description of a given table.
+``\d [table name]`` command shows the description of a given table.
 
 Also, you can execute SQL queries as follows: ::
 
@@ -62,7 +71,8 @@ Also, you can execute SQL queries as follows: ::
   4,  jkl,  4.5,  d
   5,  mno,  5.6,  e
 
-  tajo>
-
-
+  tajo> exit
+  bye
 
+Feel free to enjoy Tajo with SQL standards. 
+If you want to know more explanation for SQL supported by Tajo, please refer :doc:`/sql_language`.
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/incubator-tajo/blob/1069efb3/tajo-docs/src/main/sphinx/getting_started/local_setup.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/getting_started/local_setup.rst b/tajo-docs/src/main/sphinx/getting_started/local_setup.rst
index 328c612..e012f1b 100644
--- a/tajo-docs/src/main/sphinx/getting_started/local_setup.rst
+++ b/tajo-docs/src/main/sphinx/getting_started/local_setup.rst
@@ -2,7 +2,11 @@
 Setting up a local Tajo cluster
 **********************************
 
-First of all, you need to add the environment variables to conf/tajo-env.sh. ::
+Apache Tajo™ provides two run modes: local mode and fully distributed mode. Here, we explain
only the local mode where a Tajo instance runs on a local file system. A local mode Tajo instance
can start up with very simple configurations.
+
+First of all, you need to add the environment variables to conf/tajo-env.sh.
+
+.. code-block:: bash
 
   # Hadoop home. Required
   export HADOOP_HOME= ...
@@ -10,13 +14,12 @@ First of all, you need to add the environment variables to conf/tajo-env.sh.
::
   # The java implementation to use.  Required.
   export JAVA_HOME= ...
 
-To launch the tajo master, execute start-tajo.sh. ::
+To launch the tajo master, execute start-tajo.sh.
 
-  $ $TAJO_HOME/bin/start-tajo.sh
+.. code-block:: bash
 
-After then, you can use tsql, which is the command line shell of Tajo. ::
+  $ $TAJO_HOME/bin/start-tajo.sh
 
-  $ $TAJO_HOME/bin/tsql
-  tajo>
+.. note::
 
-If you want to how to use tsql, read Tajo Interactive Shell document.
\ No newline at end of file
+  If you want to how to setup a fully distributed mode of Tajo, please see :doc:`/configuration/cluster_setup`.


Mime
View raw message