Return-Path: X-Original-To: apmail-tajo-commits-archive@minotaur.apache.org Delivered-To: apmail-tajo-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 98F08107EF for ; Sat, 25 Jan 2014 04:56:24 +0000 (UTC) Received: (qmail 17049 invoked by uid 500); 25 Jan 2014 04:56:24 -0000 Delivered-To: apmail-tajo-commits-archive@tajo.apache.org Received: (qmail 17012 invoked by uid 500); 25 Jan 2014 04:56:23 -0000 Mailing-List: contact commits-help@tajo.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tajo.incubator.apache.org Delivered-To: mailing list commits@tajo.incubator.apache.org Received: (qmail 17005 invoked by uid 99); 25 Jan 2014 04:56:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 25 Jan 2014 04:56:23 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 25 Jan 2014 04:56:16 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 2F90A23889DA; Sat, 25 Jan 2014 04:55:54 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1561255 [2/2] - /incubator/tajo/site/ Date: Sat, 25 Jan 2014 04:55:53 -0000 To: commits@tajo.incubator.apache.org From: hyunsik@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20140125045554.2F90A23889DA@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Added: incubator/tajo/site/review-request-tool.html URL: http://svn.apache.org/viewvc/incubator/tajo/site/review-request-tool.html?rev=1561255&view=auto ============================================================================== --- incubator/tajo/site/review-request-tool.html (added) +++ incubator/tajo/site/review-request-tool.html Sat Jan 25 04:55:52 2014 @@ -0,0 +1,596 @@ + + + + + + + + + Tajo - A Big Data Warehouse System on Hadoop - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + +
+
+ +
+ + +
+ + +
+

Tajo Review Request Tool

+
+

Setup

+ +
    + +
  1. Follow instructions here to setup the jira-python package
  2. + +
  3. Follow instructions here to setup the reviewboard python tools
  4. + +
  5. Install the argparse module
  6. +
+

Linux

+ +
+
sudo yum install python-argparse
+
+

Mac

+ +
+
sudo easy_install argparse
+
+
+

Usage

+ +
+
$ ./request-patch-review.py --help
+usage: request-patch-review.py [-h] -b BRANCH -j JIRA [-skip-rb] [-s SUMMARY]
+                               [-d DESCRIPTION] [-c CHANGE_DESCRIPTION] [-pa]
+                               [-r REVIEWBOARD] [-t TESTING] [-db]
+
+Kafka patch review tool
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -b BRANCH, --branch BRANCH
+                        Tracking branch to create diff against
+  -j JIRA, --jira JIRA  JIRA corresponding to the reviewboard
+  -skip-rb, --skip-reviewboard
+                        Skip a review request to reviewboard.
+  -s SUMMARY, --summary SUMMARY
+                        Summary for the reviewboard
+  -d DESCRIPTION, --description DESCRIPTION
+                        Description for reviewboard
+  -c CHANGE_DESCRIPTION, --change-description CHANGE_DESCRIPTION
+                        Description of what changed in this revision of the
+                        review request when updating an existing request
+  -pa, --patch-available
+                        Transite the JIRA status to Patch Available. If its
+                        status is already Patch Available, it updates the
+                        status of the JIRA issue by transiting its status to
+                        Open and Patch Available sequentially.
+  -r REVIEWBOARD, --rb REVIEWBOARD
+                        Review board that needs to be updated
+  -t TESTING, --testing-done TESTING
+                        Text for the Testing Done section of the reviewboard
+  -db, --debug          Enable debug mode
+
+
+
+

Upload the first patch

+

You should create a jira issue prior to a patch review request.

+ +
    + +
  • Specify the branch against which the patch should be created (-b)
  • + +
  • Specify the corresponding JIRA (-j)
  • + +
  • Specify an optional summary (-s) and description (-d) for the reviewboard
  • + +
  • If they are not given, the summary and description of the first review request in reviewboard are borrowed from the corresponding JIRA issue.
  • +
+

Example:

+ +
+
$ ./request-patch-review.py -b origin/master -j TAJO-543
+
+
+

Update the existing patch

+ +
    + +
  • Specify the remote branch against which the patch should be created (-b)
  • + +
  • Specify the corresponding JIRA (–jira)
  • + +
  • Specify the rb to be updated (-r)
  • + +
  • Specify an optional summary (-s) and description (-d) for the reviewboard, if you want to update them.
  • + +
  • Specify a change description (-c) for the reviewboard, if you want to describe it. The change description is what changed in this revision of the review request.
  • +
+

Example:

+ +
+
$ ./request-patch-review.py -b origin/master -j TAJO-543 -r 14081 -c "Add more unit tests"
+
+
+

Preparation

+
+

JIRA command line tool

+ +
    + +
  1. Download the JIRA command line package
  2. +
+

Install the jira-python package

+ +
+
sudo easy_install jira-python
+
+ +
    + +
  1. Configure JIRA username and password
  2. +
+

Create a ${HOME}/.jira.ini file in your $HOME directory that contains your Apache JIRA username and password

+ +
+
$ cat ~/.jira.ini
+user=hyunsik
+password=***********
+
+
+

Reviewboard

+

This is a quick tutorial on using Review Board with Tajo.

+
+

1. Install the post-review tool

+

If you are on RHEL, Fedora or CentOS, follow these steps

+ +
+
sudo yum install python-setuptools
+sudo easy_install -U RBTools
+
+

If you are on Mac, follow these steps

+ +
+
sudo easy_install -U setuptools
+sudo easy_install -U RBTools
+
+

For other platforms, follow the instructions here to setup the post-review tool.

+
+

2. Configure Stuff

+

Then you need to configure a few things to make it work: First set the review board url to use. You can do this from in git:

+ +
+
git config reviewboard.url https://reviews.apache.org
+
+

If you checked out using the git wip http url that confusingly won’t work with review board. So you need to configure an override to use the non-http url. You can do this by adding a config file like this:

+ +
+
$ cat ~/.reviewboardrc
+REPOSITORY = 'git://git.apache.org/incubator-tajo.git'
+TARGET_GROUPS = 'Tajo'
+
+
+

FAQ

+

When I run the script, it throws the following error and exits

+ +
+
$ tajo-patch-review.py -b asf/master -j TAJO-593
+There don't seem to be any diffs
+
+

There are 2 reasons that can cause this - * The code is not checked into your local branch * The -b branch is not pointing to the remote branch. In the example above, “master” is specified as the branch, which is the local branch. The correct value for the -b (–branch) option is the remote branch. “git branch -r” gives the list of the remote branch names.

+

When I run the script, it throws the following error and exits

+ +
+
Error uploading diff
+
+Your review request still exists, but the diff is not attached.
+
+

One of the most common root causes of this error are that the git remote branches are not up-to-date. Since the script already does that, it is probably due to some other problem. You can run the script with the –debug option that will make post-review run in the debug mode and list the root cause of the issue.

+
+
+
+ +
+ +
+
+
Copyright © 2014 + Apache Software Foundation. + All Rights Reserved. + +
+ + +
Apache Tajo, Apache Hadoop, Apache, the Apache feather logo, and the Apache incubator logo are + trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks + or registered trademarks of their respective owners.
+ + +
+
+ + Modified: incubator/tajo/site/source-code.html URL: http://svn.apache.org/viewvc/incubator/tajo/site/source-code.html?rev=1561255&r1=1561254&r2=1561255&view=diff ============================================================================== --- incubator/tajo/site/source-code.html (original) +++ incubator/tajo/site/source-code.html Sat Jan 25 04:55:52 2014 @@ -1,13 +1,13 @@ - + Tajo - A Big Data Warehouse System on Hadoop - @@ -85,7 +85,7 @@
  • Team
  • -
  • Mailing Lists +
  • Mailing Lists
  • Issue Tracker @@ -125,6 +125,9 @@
  • Review Board
  • +
  • Review Request Tool +
  • +
  • Coding Style
  • @@ -186,7 +189,7 @@ -
  • Last Published: 2014-01-15
  • |
  • +
  • Last Published: 2014-01-25
  • |
  • Version: 0.8.0-SNAPSHOT
  • @@ -246,7 +249,7 @@
  • - + Mailing Lists
  • @@ -316,6 +319,13 @@
  • + + + Review Request Tool +
  • + +
  • + Coding Style Modified: incubator/tajo/site/source-repository.html URL: http://svn.apache.org/viewvc/incubator/tajo/site/source-repository.html?rev=1561255&r1=1561254&r2=1561255&view=diff ============================================================================== --- incubator/tajo/site/source-repository.html (original) +++ incubator/tajo/site/source-repository.html Sat Jan 25 04:55:52 2014 @@ -1,13 +1,13 @@ - + Tajo - A Big Data Warehouse System on Hadoop - Source Repository @@ -85,7 +85,7 @@
  • Team
  • -
  • Mailing Lists +
  • Mailing Lists
  • Issue Tracker @@ -125,6 +125,9 @@
  • Review Board
  • +
  • Review Request Tool +
  • +
  • Coding Style
  • @@ -186,7 +189,7 @@ -
  • Last Published: 2014-01-15
  • |
  • +
  • Last Published: 2014-01-25
  • |
  • Version: 0.8.0-SNAPSHOT
  • @@ -246,7 +249,7 @@
  • - + Mailing Lists
  • @@ -318,6 +321,13 @@
  • + + + Review Request Tool +
  • + +
  • + Coding Style Modified: incubator/tajo/site/surefire-report.html URL: http://svn.apache.org/viewvc/incubator/tajo/site/surefire-report.html?rev=1561255&r1=1561254&r2=1561255&view=diff ============================================================================== --- incubator/tajo/site/surefire-report.html (original) +++ incubator/tajo/site/surefire-report.html Sat Jan 25 04:55:52 2014 @@ -1,13 +1,13 @@ - + Tajo - A Big Data Warehouse System on Hadoop - Surefire Report @@ -85,7 +85,7 @@
  • Team
  • -
  • Mailing Lists +
  • Mailing Lists
  • Issue Tracker @@ -125,6 +125,9 @@
  • Review Board
  • +
  • Review Request Tool +
  • +
  • Coding Style
  • @@ -186,7 +189,7 @@ -
  • Last Published: 2014-01-15
  • |
  • +
  • Last Published: 2014-01-25
  • |
  • Version: 0.8.0-SNAPSHOT
  • @@ -246,7 +249,7 @@
  • - + Mailing Lists
  • @@ -318,6 +321,13 @@
  • + + + Review Request Tool +
  • + +
  • + Coding Style Modified: incubator/tajo/site/tajo-0.2.0-doc.html URL: http://svn.apache.org/viewvc/incubator/tajo/site/tajo-0.2.0-doc.html?rev=1561255&r1=1561254&r2=1561255&view=diff ============================================================================== --- incubator/tajo/site/tajo-0.2.0-doc.html (original) +++ incubator/tajo/site/tajo-0.2.0-doc.html Sat Jan 25 04:55:52 2014 @@ -1,13 +1,13 @@ - + Tajo - A Big Data Warehouse System on Hadoop - @@ -85,7 +85,7 @@
  • Team
  • -
  • Mailing Lists +
  • Mailing Lists
  • Issue Tracker @@ -125,6 +125,9 @@
  • Review Board
  • +
  • Review Request Tool +
  • +
  • Coding Style
  • @@ -186,7 +189,7 @@ -
  • Last Published: 2014-01-15
  • |
  • +
  • Last Published: 2014-01-25
  • |
  • Version: 0.8.0-SNAPSHOT
  • @@ -246,7 +249,7 @@
  • - + Mailing Lists
  • @@ -316,6 +319,13 @@
  • + + + Review Request Tool +
  • + +
  • + Coding Style Modified: incubator/tajo/site/tajo-0.8.0-doc.html URL: http://svn.apache.org/viewvc/incubator/tajo/site/tajo-0.8.0-doc.html?rev=1561255&r1=1561254&r2=1561255&view=diff ============================================================================== --- incubator/tajo/site/tajo-0.8.0-doc.html (original) +++ incubator/tajo/site/tajo-0.8.0-doc.html Sat Jan 25 04:55:52 2014 @@ -1,13 +1,13 @@ - + Tajo - A Big Data Warehouse System on Hadoop - @@ -85,7 +85,7 @@
  • Team
  • -
  • Mailing Lists +
  • Mailing Lists
  • Issue Tracker @@ -125,6 +125,9 @@
  • Review Board
  • +
  • Review Request Tool +
  • +
  • Coding Style
  • @@ -186,7 +189,7 @@ -
  • Last Published: 2014-01-15
  • |
  • +
  • Last Published: 2014-01-25
  • |
  • Version: 0.8.0-SNAPSHOT
  • @@ -246,7 +249,7 @@
  • - + Mailing Lists
  • @@ -316,6 +319,13 @@
  • + + + Review Request Tool +
  • + +
  • + Coding Style @@ -933,8 +943,68 @@ $ $HADOOP_HOME/bin/hadoop fs -chmod g+w
  • tajo.catalog.store.MySQLStore - this storage class uses MySQL.
  • tajo.catalog.store.MemStore - this is the in-memory storage. It is only used in unit tests to shorten the duration of unit tests.
  • + +
  • tajo.catalog.store.HCatalogStore - this storage class uses HiveMetaStore.
  • - + +
    +

    MySQLStore Configuration

    +

    If you want to use MySQLStore, you must create database and user on mysql for tajo.

    +

    And then, you need to prepare mysql jdbc driver on host which can be ran TajoMaster. If you do, you should set TAJO_CLASSPATH variable in conf/tajo-env.sh with it as follows:

    + +
    +
    export TAJO_CLASSPATH=/usr/local/mysql/lib/mysql-connector-java-x.x.x.jar
    +
    +

    Or you just can copy jdbc driver into $TAJO_HOME/lib.

    +

    Finally, you should add the following config to conf/catalog-site.xml :

    + +
    +
      <property>
    +    <name>tajo.catalog.store.class</name>
    +    <value>org.apache.tajo.catalog.store.MySQLStore</value>
    +  </property>
    +  <property>
    +    <name>tajo.catalog.jdbc.connection.id</name>
    +    <value><mysql user name></value>
    +  </property>
    +  <property>
    +    <name>tajo.catalog.jdbc.connection.password</name>
    +    <value><mysql user password></value>
    +  </property>
    +    <property>
    +    <name>tajo.catalog.jdbc.uri</name>
    +    <value>jdbc:mysql://<mysql host name>:<mysql port>/<database name for tajo>?createDatabaseIfNotExist=true</value>
    +  </property>
    +
    +
    +

    HCatalogStore Configuration

    +

    Tajo support HCatalogStore to integrate with hive. If you want to use HCatalogStore, you just do as follows.

    +

    First, you must compile source code and get a binary archive as follows:

    + +
    +
    $ git clone https://git-wip-us.apache.org/repos/asf/incubator-tajo.git tajo
    +$ mvn clean package -DskipTests -Pdist -Dtar -Phcatalog-0.1x.0
    +$ ls tajo-dist/target/tajo-0.8.0-SNAPSHOT.tar.gz
    +
    +

    Tajo support to build based on hive 0.11.0 and hive 0.12.0. If you use hive 0.11.0, you have to set -Phcatalog-0.11.0. And if you use hive 0.12.0, you have to set -Phcatalog-0.12.0.

    +

    Second, you must set your hive home directory to HIVE_HOME variable in conf/tajo-env.sh with it as follows:

    + +
    +
    export HIVE_HOME=/path/to/your/hive/directory
    +
    +

    Third, if you need to use jdbc to connect HiveMetaStore, you have to prepare mysql jdbc driver on host which can be ran TajoMaster. If you prepare it, you should set jdbc driver file path to HIVE_JDBC_DRIVER_DIR variable in conf/tajo-env.sh with it as follows:

    + +
    +
    export HIVE_JDBC_DRIVER_DIR==/path/to/your/mysql_jdbc_driver/mysql-connector-java-x.x.x-bin.jar
    +
    +

    Lastly, you should add the following config to conf/catalog-site.xml :

    + +
    +
      <property>
    +    <name>tajo.catalog.store.class</name>
    +    <value>org.apache.tajo.catalog.store.HCatalogStore</value>
    +  </property>
    +

    RPC/Http Service Configuration and Default Addresses

    Modified: incubator/tajo/site/team-list.html URL: http://svn.apache.org/viewvc/incubator/tajo/site/team-list.html?rev=1561255&r1=1561254&r2=1561255&view=diff ============================================================================== --- incubator/tajo/site/team-list.html (original) +++ incubator/tajo/site/team-list.html Sat Jan 25 04:55:52 2014 @@ -1,13 +1,13 @@ - + Tajo - A Big Data Warehouse System on Hadoop - Team list @@ -85,7 +85,7 @@
  • Team
  • -
  • Mailing Lists +
  • Mailing Lists
  • Issue Tracker @@ -125,6 +125,9 @@
  • Review Board
  • +
  • Review Request Tool +
  • +
  • Coding Style
  • @@ -186,7 +189,7 @@ -
  • Last Published: 2014-01-15
  • |
  • +
  • Last Published: 2014-01-25
  • |
  • Version: 0.8.0-SNAPSHOT
  • @@ -244,7 +247,7 @@
  • - + Mailing Lists
  • @@ -316,6 +319,13 @@
  • + + + Review Request Tool +
  • + +
  • + Coding Style @@ -442,7 +452,7 @@ Keuntae Park keuntae.park@sk.com SK Telecom -Committer +- +9 +9