tajo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jihoon...@apache.org
Subject tajo git commit: TAJO-1855: Add document for tablespace configuration to the Configuration chapter.
Date Wed, 02 Dec 2015 12:41:46 GMT
Repository: tajo
Updated Branches:
  refs/heads/master 0af53d025 -> f2552bfb9


TAJO-1855: Add document for tablespace configuration to the Configuration chapter.

Closes #873

Signed-off-by: Jihoon Son <jihoonson@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/tajo/repo
Commit: http://git-wip-us.apache.org/repos/asf/tajo/commit/f2552bfb
Tree: http://git-wip-us.apache.org/repos/asf/tajo/tree/f2552bfb
Diff: http://git-wip-us.apache.org/repos/asf/tajo/diff/f2552bfb

Branch: refs/heads/master
Commit: f2552bfb9c0a3b57f66e89a9e635a0d1afbeebc8
Parents: 0af53d0
Author: Jongyoung Park <eminency@gmail.com>
Authored: Wed Dec 2 21:41:13 2015 +0900
Committer: Jihoon Son <jihoonson@apache.org>
Committed: Wed Dec 2 21:41:13 2015 +0900

----------------------------------------------------------------------
 CHANGES                                         |  3 +
 tajo-docs/src/main/sphinx/conf.py               |  2 +-
 tajo-docs/src/main/sphinx/configuration.rst     |  7 ++-
 .../sphinx/configuration/storage-site-json.rst  | 64 ++++++++++++++++++++
 .../sphinx/table_management/tablespaces.rst     | 12 ++--
 5 files changed, 80 insertions(+), 8 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/CHANGES
----------------------------------------------------------------------
diff --git a/CHANGES b/CHANGES
index 86ed3cd..e0fd693 100644
--- a/CHANGES
+++ b/CHANGES
@@ -92,6 +92,9 @@ Release 0.12.0 - unreleased
 
   TASKS
 
+    TAJO-1855: Add document for tablespace configuration to the Configuration chapter.
+    (Contributed by Jongyoung Park. Committed by jihoon)
+
     TAJO-1854: Add in-subquery document. (jihoon)
 
     TAJO-2003: Fix typo in worker configuration. (jinho)

http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/tajo-docs/src/main/sphinx/conf.py
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/conf.py b/tajo-docs/src/main/sphinx/conf.py
index f6a2d96..dbd9e14 100644
--- a/tajo-docs/src/main/sphinx/conf.py
+++ b/tajo-docs/src/main/sphinx/conf.py
@@ -58,7 +58,7 @@ master_doc = 'index'
 
 # General information about the project.
 project = u'Apache Tajo'
-copyright = u'2014, Apache Tajo Team'
+copyright = u'2015, Apache Tajo Team'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the

http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/tajo-docs/src/main/sphinx/configuration.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/configuration.rst b/tajo-docs/src/main/sphinx/configuration.rst
index 5214ea8..8b0b8a2 100644
--- a/tajo-docs/src/main/sphinx/configuration.rst
+++ b/tajo-docs/src/main/sphinx/configuration.rst
@@ -1,6 +1,6 @@
-************************
+*************
 Configuration
-************************
+*************
 
 .. toctree::
     :maxdepth: 1
@@ -13,4 +13,5 @@ Configuration
     configuration/ha_configuration
     configuration/service_config_defaults
     configuration/tajo-site-xml
-    configuration/catalog-site-xml
\ No newline at end of file
+    configuration/catalog-site-xml
+    configuration/storage-site-json

http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/tajo-docs/src/main/sphinx/configuration/storage-site-json.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/configuration/storage-site-json.rst b/tajo-docs/src/main/sphinx/configuration/storage-site-json.rst
new file mode 100644
index 0000000..d377551
--- /dev/null
+++ b/tajo-docs/src/main/sphinx/configuration/storage-site-json.rst
@@ -0,0 +1,64 @@
+**************************
+The storage-site.json File
+**************************
+
+This file is for configuring :doc:`/table_management/tablespaces`.
+The syntax of ``storage-site.json`` is like this basically:
+
+.. code:: json
+
+  {
+    "spaces": {
+      <space config 1>, <space config 2>, ...
+    },
+    "storages": {
+      <storage config 1>, <storage config 2>, ...
+    }
+  }
+
+------
+Spaces
+------
+
+This is a section for registering tablespaces. Some space config example is here:
+
+.. code:: json
+
+  "spaces": {
+    "jdbc_pgsql": {
+      "uri": "jdbc:postgresql://127.0.0.1:5740/origin",
+      "configs": {
+        "mapped_database":"tajo_pgsql_origin",
+        "connection_properties": {
+          "user":"tajouser",
+          "password":"123456"
+        }
+      }
+    },
+    ...
+  }
+
+* **space name** : Your own tablespace name which indicates a specific tablespace. Alpha-numeric
characters and underscore(_) are permitted.
+* **uri** : An URI address of a tablespace
+* **configs** : It is optional. You can specify it as JSON object to pass to each tablespace
handler.
+
+After you specify a tablespace, you can use it in `create table statement <../sql_language/ddl.html#create-table>`_.
+
+--------
+Storages
+--------
+
+This is for registering storage format and custom storage handler class.
+Tajo already supports HDFS, HBase, PostgreSQL, Amazon S3, Openstack Swift, etc, thus in usual
cases using mentioned storages, you don't have to add any ``storages`` configuration.
+However, if you want to use your custom storage as one of Tajo's data source, you need to
add a configuration for your storage. Here is an example of HDFS storage.
+See :doc:`storage_plugins/overview` for more information.
+
+.. code:: json
+
+  "storages": {
+    "hdfs": {
+      "handler": "org.apache.tajo.storage.FileTablespace",
+      "default-format": "text"
+    },
+    ...
+  }

http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/tajo-docs/src/main/sphinx/table_management/tablespaces.rst
----------------------------------------------------------------------
diff --git a/tajo-docs/src/main/sphinx/table_management/tablespaces.rst b/tajo-docs/src/main/sphinx/table_management/tablespaces.rst
index 6f43af1..ff90483 100644
--- a/tajo-docs/src/main/sphinx/table_management/tablespaces.rst
+++ b/tajo-docs/src/main/sphinx/table_management/tablespaces.rst
@@ -26,9 +26,11 @@ If the tablespace is not specified, the default tablespace of the table's
databa
 Configuration
 =============
 
-By default, Tajo use in ``${tajo.rootdir}/warehouse`` in ``conf/tajo-site.xml`` as a default
tablespace. It also allows users to register additional tablespaces. 
+By default, Tajo use in ``${tajo.rootdir}/warehouse`` in :doc:`conf/tajo-site.xml</configuration/tajo-site-xml>`
as a default tablespace. It also allows users to register additional tablespaces using ``storage-site.json``
file like below.
 
-``conf/storage-site.json`` file.
+---------------------------
+conf/storage-site.json file
+---------------------------
 
 The configuration file has the following struct:
 
@@ -36,7 +38,7 @@ The configuration file has the following struct:
 
   {
     "spaces": {
-      "${table_space_name}": {
+      "${tablespace_name}": {
         "uri": "hbase://quorum1:port,quorum2:port/"
       }
     }
@@ -48,7 +50,7 @@ The following is an example for two tablespaces for hbase and hdfs:
 
   {
     "spaces": {
-      "hbase-cluster1": {
+      "hbase_cluster1": {
         "uri": "hbase://quorum1:port,quorum2:port/"
       },
 
@@ -58,6 +60,8 @@ The following is an example for two tablespaces for hbase and hdfs:
     }
   }
 
+For more details, see :doc:`conf/storage-site.json</configuration/storage-site-json>`.
+
 
 .. note::
 


Mime
View raw message