Return-Path: X-Original-To: apmail-kylin-commits-archive@minotaur.apache.org Delivered-To: apmail-kylin-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BB53318293 for ; Fri, 14 Aug 2015 03:21:26 +0000 (UTC) Received: (qmail 51053 invoked by uid 500); 14 Aug 2015 03:21:26 -0000 Delivered-To: apmail-kylin-commits-archive@kylin.apache.org Received: (qmail 51025 invoked by uid 500); 14 Aug 2015 03:21:26 -0000 Mailing-List: contact commits-help@kylin.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kylin.incubator.apache.org Delivered-To: mailing list commits@kylin.incubator.apache.org Received: (qmail 50948 invoked by uid 99); 14 Aug 2015 03:21:26 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Aug 2015 03:21:26 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 23753C63BC for ; Fri, 14 Aug 2015 03:21:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.754 X-Spam-Level: **** X-Spam-Status: No, score=4.754 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_BADIPHTTP=2, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-0.006, URIBL_BLOCKED=0.001, URI_TRY_3LD=0.958, WEIRD_PORT=0.001] autolearn=disabled Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id a-CtZRmjVDkK for ; Fri, 14 Aug 2015 03:21:11 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with SMTP id D353142BEC for ; Fri, 14 Aug 2015 03:21:07 +0000 (UTC) Received: (qmail 49025 invoked by uid 99); 14 Aug 2015 03:21:06 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Aug 2015 03:21:06 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 83132E7145; Fri, 14 Aug 2015 03:21:06 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: lukehan@apache.org To: commits@kylin.incubator.apache.org Date: Fri, 14 Aug 2015 03:21:23 -0000 Message-Id: In-Reply-To: <820dc0eac4064a88ba9e3bda67c222ac@git.apache.org> References: <820dc0eac4064a88ba9e3bda67c222ac@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [18/50] [abbrv] incubator-kylin git commit: KYLIN-907 Correct some wrontly classified howto/dev articles KYLIN-907 Correct some wrontly classified howto/dev articles Project: http://git-wip-us.apache.org/repos/asf/incubator-kylin/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-kylin/commit/6d59b249 Tree: http://git-wip-us.apache.org/repos/asf/incubator-kylin/tree/6d59b249 Diff: http://git-wip-us.apache.org/repos/asf/incubator-kylin/diff/6d59b249 Branch: refs/heads/0.7 Commit: 6d59b249a2c17937c09f8bf309bf5e6dbcbcd760 Parents: f2c4568 Author: Li, Yang Authored: Fri Aug 7 17:08:59 2015 +0800 Committer: Li, Yang Committed: Fri Aug 7 17:09:35 2015 +0800 ---------------------------------------------------------------------- website/_data/development.yml | 8 +- website/_data/docs.yml | 10 +- website/_dev/build_api.md | 56 - website/_dev/howto_contribute.md | 28 + website/_dev/howto_docs.md | 145 ++ website/_dev/howto_package.md | 13 + website/_dev/howto_release.md | 449 ++++ website/_dev/javascript_api.md | 47 - website/_dev/metadata_store.md | 61 - website/_dev/rest_api.md | 2031 ----------------- website/_docs/howto/howto_backup.md | 26 - website/_docs/howto/howto_backup_hbase.md | 26 + website/_docs/howto/howto_backup_metadata.md | 62 + .../howto/howto_build_cube_with_restapi.md | 58 + website/_docs/howto/howto_contribute.md | 30 - website/_docs/howto/howto_docs.md | 147 -- website/_docs/howto/howto_package.md | 15 - website/_docs/howto/howto_release.md | 451 ---- website/_docs/howto/howto_use_restapi.md | 2033 ++++++++++++++++++ website/_docs/howto/howto_use_restapi_in_js.md | 49 + website/_docs/install/index.cn.md | 16 +- website/_docs/install/index.md | 12 +- 22 files changed, 2886 insertions(+), 2887 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_data/development.yml ---------------------------------------------------------------------- diff --git a/website/_data/development.yml b/website/_data/development.yml index 217c0b7..50ac6a4 100644 --- a/website/_data/development.yml +++ b/website/_data/development.yml @@ -20,11 +20,11 @@ dev: - index - dev_env - - metadata_store + - howto_contribute + - howto_docs + - howto_package + - howto_release - new_metadata - - rest_api - - build_api - - javascript_api - web_tech - test_minicluster \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_data/docs.yml ---------------------------------------------------------------------- diff --git a/website/_data/docs.yml b/website/_data/docs.yml index 7d03a64..f747579 100644 --- a/website/_data/docs.yml +++ b/website/_data/docs.yml @@ -44,10 +44,10 @@ - title: How To docs: - - howto/howto_contribute + - howto/howto_build_cube_with_restapi + - howto/howto_use_restapi_in_js + - howto/howto_use_restapi + - howto/howto_backup_metadata + - howto/howto_backup_hbase - howto/howto_jdbc - - howto/howto_javascript - - howto/howto_backup - howto/howto_upgrade - - howto/howto_docs - - howto/howto_release http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_dev/build_api.md ---------------------------------------------------------------------- diff --git a/website/_dev/build_api.md b/website/_dev/build_api.md deleted file mode 100644 index 62c7d1d..0000000 --- a/website/_dev/build_api.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -layout: dev -title: "Build Cube with Restful API" -categories: development -permalink: /development/build_api.html ---- - -### 1. Authentication -* Currently, Kylin uses [basic authentication](http://en.wikipedia.org/wiki/Basic_access_authentication). -* Add `Authorization` header to first request for authentication -* Or you can do a specific request by `POST http://localhost:7070/kylin/api/user/authentication` -* Once authenticated, client can go subsequent requests with cookies. -* Example -{% highlight Groff markup %} - POST http://localhost:7070/kylin/api/user/authentication - - Authorization:Basic xxxxJD124xxxGFxxxSDF - Content-Type: application/json;charset=UTF-8 -{% endhighlight %} - -### 2. Get details of cube. -* `GET http://localhost:7070/kylin/api/cubes?cubeName={cube_name}&limit=15&offset=0` -* Client can find cube segment date ranges in returned cube detail. -* Example -{% highlight Groff markup %} - GET http://localhost:7070/kylin/api/cubes?cubeName=test_kylin_cube_with_slr&limit=15&offset=0 - - Authorization:Basic xxxxJD124xxxGFxxxSDF - Content-Type: application/json;charset=UTF-8 -{% endhighlight %} -### 3. Then submit a build job of the cube. -* `PUT http://localhost:7070/kylin/api/cubes/{cube_name}/rebuild` -* For put request body detail please refer to [service doc](rest_api.html). - * `startTime` and `endTime` should be utc timestamp. - * `buildType` can be `BUILD` ,`MERGE` or `REFRESH`. `BUILD` is for building a new segment, `REFRESH` for refreshing an existing segment. `MERGE` is for merging multiple existing segments into one bigger segment. -* This method will return a newly created job instance, in which the uuid is the identity of job to track job status. -* Example -{% highlight Groff markup %} - PUT http://localhost:7070/kylin/api/cubes/test_kylin_cube_with_slr/rebuild - - Authorization:Basic xxxxJD124xxxGFxxxSDF - Content-Type: application/json;charset=UTF-8 - - { - "startTime": 0, - "endTime": 1388563200000, - "buildType": "BUILD" - } -{% endhighlight %} - -### 4. Track job status. -* `GET http://localhost:7070/kylin/api/jobs/{job_uuid}` -* Returned `job_status` represents current status of job. - -### 5. If the job failed, you can resume the job. -* `PUT http://localhost:7070/kylin/api/jobs/{job_uuid}/resume` http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_dev/howto_contribute.md ---------------------------------------------------------------------- diff --git a/website/_dev/howto_contribute.md b/website/_dev/howto_contribute.md new file mode 100644 index 0000000..bb35956 --- /dev/null +++ b/website/_dev/howto_contribute.md @@ -0,0 +1,28 @@ +--- +layout: dev +title: How to Contribute +categories: development +permalink: /development/howto_contribute.html +--- + +## Current branches + +Here are the major branches: + +* 0.7-staging: dev branch for 0.7 versions, this branch spawns releases like 0.7.1, 0.7.2, the next release would be 0.7.3. +* 0.8: dev branch for 0.8 versions, 0.8 is like the next generation Kylin (with streaming, spark support), it has fundamental difference with 0.7 version, which means any changes on 0.7 will not be merged to 0.8 anymore. So if your patch affects both of the branch, you should make patches for both branch. +* master: always point to the latest stable release(stable, but not up to date) + +## Making Changes +* Raise an issue on JIRA, describe the feature/enhancement/bug +* Discuss with others in mailing list or issue comments, make sure the proposed changes fit in with what others are doing and have planned for the project +* Make changes in your fork +* Write unit test if no existing cover your change +* Generate patch and attach it to relative JIRA (https://ariejan.net/2009/10/26/how-to-create-and-apply-a-patch-with-git/) + +## Apply Patch +* Committer will review in terms of correctness, performance, design, coding style, test coverage +* Discuss and revise if necessary +* Finally committer merge code into main branch + + http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_dev/howto_docs.md ---------------------------------------------------------------------- diff --git a/website/_dev/howto_docs.md b/website/_dev/howto_docs.md new file mode 100644 index 0000000..426aefd --- /dev/null +++ b/website/_dev/howto_docs.md @@ -0,0 +1,145 @@ +--- +layout: dev +title: How to Write Document +categories: development +permalink: /development/howto_docs.html +--- + +## Before your work + +Install following tools before you add or edit documentation: +1. First, make sure Ruby and Gem are works on your machine +_For Mac User, please refer [this](https://github.com/sstephenson/rbenv#homebrew-on-mac-os-x) to setup ruby env._ +2. Then, install [Jekyll](http://jekyllrb.com), and plugins, for example: +`gem install jekyll jekyll-multiple-languages kramdown rouge` +3. [Jekyll Multiple Languages Plugin](http://jekyll-langs.liaohuqiu.net/cn/) +4. And any markdown editor + +## About Jekyll +Jekyll is a Ruby script to generate a static HTML website from source text and themes, the HTML is generated before being deployed to the web server. Jekyll also happens to be the engine behind GitHub Pages. + +Here are good reference about basic usage of Jekyll: [Learning Jekyll By Example](http://learn.andrewmunsell.com/learn/jekyll-by-example/tutorial) + +Apache Kylin's website and documentation is using Jekyll to manage and generate final content which avaliable at [http://kylin.incubator.apache.org](http://kylin.incubator.apache.org). + +## Multi-Language +To draft Chinese version document or translate existing one, just add or copy that doc and name with .cn.md as sufffix. It will generate under /cn folder with same name as html file. +To add other language, please update _config.yml and follow the same pattern as Chinese version. + +# Kylin document sturcture and navigation menu + +1. __Home Page__: _"index.md"_ Home page of Docs +2. __Getting Started__: _"gettingstarted"_ General docs about Apache Kylin, including FAQ, Terminology +3. __Installation__: _"install"_ Apache Kylin installation guide +4. __Tutorial__: _"tutorial"_ User tutorial about how to use Apache Kylin +5. __How To__: _"howto"_ Guide for more detail help +6. __Development__: _"development"_ For developer to contribute, integration with other application and extend Apache Kylin +7. __Others__: Other docs. + +The menu is managed by Jekyll collection: + +* ___data/docs.yml__: English version menu structure +* ___data/docs-cn.yml__: Chinese version menu structure +* __add new menu item__: To add new item: create new docs under relative folder, e.g howto_example.md. add following Front Mark: + +``` +--- +layout: docs +title: How to expamle +categories: howto +permalink: /docs/howto/howto_example.html +version: v0.7.2 +since: v0.7.2 +--- +``` + +change the __permalink__ to exactly link +Then add item to docs.yml like: + +``` +- title: How To + docs: + - howto/howto_contribute + - howto/howto_jdbc + - howto/howto_example +``` + +# How to edit document +Open doc with any markdown editor, draft content and preview in local. + +Sample Doc: + +``` +--- +layout: docs +title: How to example +categories: howto +permalink: /docs/howto/howto_example.html +version: v0.7.2 +since: v0.7.2 +--- + +## This is example doc +The quick brown fox jump over the lazy dog. + +``` + +# How to add image +All impage please put under _images_ folder, in your document, please using below sample to include image: + +``` +![](/images/Kylin-Web-Tutorial/2 tables.png) + +``` + +# How to add link +Using relative path for site links, for example: + +``` +[REST API](docs/development/rest_api.html). + +``` + +# How to add code highlight +We are using [Rouge](https://github.com/jneen/rouge) to highlight code syntax. +check this doc's source code for more detail sample. + +# How to preview in your local +You can preview in your markdown editor, to check exactly what it will looks like on website, please run Jekyll from "website" folder: +``` +jekyll s + +``` +Then access http://127.0.0.1:4000 in your browser. + +## How to publish to website (for committer only) + +### Setup + +1. `cd website` +2. `svn co https://svn.apache.org/repos/asf/incubator/kylin/site _site` +3. `sudo apt-get install rubygems ruby2.1-dev zlib1g-dev` (linux) +4. `sudo gem install bundler github-pages jekyll` +5. `bundle install` + +___site__ folder is working dir which will be removed anytime by maven or git, please make sure only check out from svn when you want to publish to website. + +### Running locally +Before opening a pull request or push to git repo, you can preview changes from your local box with following: + +1. `cd website` +2. `jekyll s` +3. Open [http://127.0.0.1:4000](http://127.0.0.1:4000) in your browser + +### Pushing to site + +1. `cd website/_site` +2. `svn status` +3. You'll need to `svn add` any new files +4. `svn commit -m 'UPDATE MESSAGE'` + +Within a few minutes, svnpubsub should kick in and you'll be able to +see the results at +[http://kylin.incubator.apache.org](http://kylin.incubator.apache.org/). + + http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_dev/howto_package.md ---------------------------------------------------------------------- diff --git a/website/_dev/howto_package.md b/website/_dev/howto_package.md new file mode 100644 index 0000000..acc0de2 --- /dev/null +++ b/website/_dev/howto_package.md @@ -0,0 +1,13 @@ +--- +layout: dev +title: How to Build Binary Package +categories: development +permalink: /development/howto_package.html +--- + +###Generate Binary Package +> git clone https://github.com/apache/incubator-kylin kylin +> cd kylin +> ./script/package.sh + +In order to generate binary package, **maven** and **npm** are pre-requisites. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_dev/howto_release.md ---------------------------------------------------------------------- diff --git a/website/_dev/howto_release.md b/website/_dev/howto_release.md new file mode 100644 index 0000000..a311a61 --- /dev/null +++ b/website/_dev/howto_release.md @@ -0,0 +1,449 @@ +--- +layout: dev +title: How to Making a Release +categories: development +permalink: /development/howto_release.html +--- + +_This guide is for Apache Kylin Committers only._ +_Shell commands is on Mac OS X as sample._ +_For people in China, please aware using proxy to avoid potential firewall issue._ + +## Setup Account +Make sure you have avaliable account and privlidge for following applications: + +* Apache account: [https://id.apache.org](https://id.apache.org/) +* Apache Kylin git repo (main cobe base): [https://git-wip-us.apache.org/repos/asf/incubator-kylin.git](https://git-wip-us.apache.org/repos/asf/incubator-kylin.git) +* Apache Kylin svn repo (for website only): [https://svn.apache.org/repos/asf/incubator/kylin](https://svn.apache.org/repos/asf/incubator/kylin) +* Apache Nexus (maven repo): [https://repository.apache.org](https://repository.apache.org) +* Apache Kylin dist repo: [https://dist.apache.org/repos/dist/dev/incubator/kylin](https://dist.apache.org/repos/dist/dev/incubator/kylin) + +## Setup PGP signing keys +Follow instructions at [http://www.apache.org/dev/release-signing](http://www.apache.org/dev/release-signing) to create a key pair +Install gpg (On Mac OS X as sample): +`brew install gpg and gpg --gen-key` + +Generate gpg key: +Reference: [https://www.gnupg.org/gph/en/manual/c14.html](https://www.gnupg.org/gph/en/manual/c14.html) +_All new RSA keys generated should be at least 4096 bits. Do not generate new DSA keys_ +`gpg --gen-key` + +Verify your key: +`gpg --list-sigs YOUR_NAME` + +Then add your key to your apache account, for example: +[https://people.apache.org/keys/committer/lukehan.asc](https://people.apache.org/keys/committer/lukehan.asc) +Generate ASCII Amromed Key: +`gpg -a --export YOUR_MAIL_ADDRESS > YOUR_NAME.asc &` + +Upload key to public server: +`gpg --send-keys YOUR_KEY_HASH` + +or Submit key via web: +Open and Submit to [http://pool.sks-keyservers.net:11371](http://pool.sks-keyservers.net:11371) (you can pickup any avaliable public key server) +Once your key submitted to server, you can verify using following command: +`gpg --recv-keys YOUR_KEY_HASH` + +for example: +`gpg --recv-keys 027DC364` + +Add your public key to the KEYS file by following instructions in the KEYS file.: +_KEYS file location:_ __${incubator-kylin}/KEYS__ +For example: +`(gpg --list-sigs YOURNAME && gpg --armor --export YOURNAME) >> KEYS` + +Commit your changes. + +## Prepare artifacts for release +__Before you start:__ + +* Set up signing keys as described above. +* Make sure you are using JDK 1.7 (not 1.8). +* Make sure you are working on right release version number. +* Make sure that every “resolved” JIRA case (including duplicates) has a fix version assigned. + +__Verify licenses__ +Run Apache RAT to check licenses issue: +{% highlight bash %} +mvn -Papache-release clean rat:rat +{% endhighlight %} + +Fix license issue if any. + +__Making a snapshot__ +{% highlight bash %} +# Set passphrase variable without putting it into shell history +$ read -s GPG_PASSPHRASE + +# Make sure that there are no junk files in the sandbox +$ git clean -xn +$ mvn clean + +$ mvn -Papache-release -Dgpg.passphrase=${GPG_PASSPHRASE} install +{% endhighlight %} +When the dry-run has succeeded, change install to deploy. + +__Making a release__ + +Create a release branch named after the release, e.g. v0.7.2-release, and push it to Apache. +{% highlight bash %} +$ git checkout -b vX.Y.Z-release +$ git push -u origin vX.Y.Z-release +{% endhighlight %} +We will use the branch for the entire the release process. Meanwhile, we do not allow commits to the master branch. After the release is final, we can use `git merge --ff-only` to append the changes on the release branch onto the master branch. (Apache does not allow reverts to the master branch, which makes it difficult to clean up the kind of messy commits that inevitably happen while you are trying to finalize a release.) + +Now, set up your environment and do a dry run. The dry run will not commit any changes back to git and gives you the opportunity to verify that the release process will complete as expected. + +If any of the steps fail, clean up (see below), fix the problem, and start again from the top. +{% highlight bash %} +# Set passphrase variable without putting it into shell history +$ read -s GPG_PASSPHRASE + +# Make sure that there are no junk files in the sandbox +$ git clean -xn +$ mvn clean + +# Do a dry run of the release:prepare step, which sets version numbers. +$ mvn -DdryRun=true -DskipTests -DreleaseVersion=X.Y.Z-incubating -DdevelopmentVersion=X.Y.Z+1-incubating-SNAPSHOT -Papache-release -Darguments="-Dgpg.passphrase=${GPG_PASSPHRASE}" release:prepare 2>&1 | tee /tmp/prepare-dry.log +{% endhighlight %} + +__Check the artifacts:__ + +* In the `target` directory should be these 8 files, among others: + * apache-kylin-X.Y.Z-incubating-src.tar.gz + * apache-kylin-X.Y.Z-incubating-src.tar.gz.asc + * apache-kylin-X.Y.Z-incubating-src.tar.gz.md5 + * apache-kylin-X.Y.Z-incubating-src.tar.gz.sha1 + * apache-kylin-X.Y.Z-incubating-src.zip + * apache-kylin-X.Y.Z-incubating-src.zip.asc + * apache-kylin-X.Y.Z-incubating-src.zip.md5 + * apache-kylin-X.Y.Z-incubating-src.zip.sha1 +* Note that the file names start `apache-kylin-` and include + `incubating` in the version. +* In the two source distros `.tar.gz` and `.zip`, check that all files belong to a directory called + `apache-kylin-X.Y.Z-incubating-src`. +* That directory must contain files `DISCLAIMER`, `NOTICE`, `LICENSE`, `README.md` +* Check PGP, per [this](https://httpd.apache.org/dev/verification.html) + +__Run real release:__ +Now, run the release for real. +{% highlight bash %} +# Prepare sets the version numbers, creates a tag, and pushes it to git. +$ mvn -DskipTests -Papache-release -Darguments="-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests" clean release:prepare + +# Perform checks out the tagged version, builds, and deploys to the staging repository +$ mvn -DskipTests -Papache-release -Darguments="-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests" release:perform +{% endhighlight %} + +__Verify the staged artifacts in the Nexus repository:__ +* Go to [https://repository.apache.org/](https://repository.apache.org/) and login +* Under `Build Promotion`, click `Staging Repositories` +* In the `Staging Repositories` tab there should be a line with profile `org.apache.kylin` +* Navigate through the artifact tree and make sure the .jar, .pom, .asc files are present +* Check the box on in the first column of the row, + and press the 'Close' button to publish the repository at + https://repository.apache.org/content/repositories/orgapachekylin-1006 + (or a similar URL) + +__Upload to staging area:__ +Upload the artifacts via subversion to a staging area, https://dist.apache.org/repos/dist/dev/incubator/kylin/apache-kylin-X.Y.Z-incubating-rcN: +{% highlight bash %} +# Create a subversion workspace, if you haven't already +$ mkdir -p ~/dist/dev +$ pushd ~/dist/dev +$ svn co https://dist.apache.org/repos/dist/dev/incubator/kylin +$ popd + +## Move the files into a directory +$ cd target +$ mkdir ~/dist/dev/kylin/apache-kylin-X.Y.Z-incubating-rcN +$ mv apache-kylin-* ~/dist/dev/kylin/apache-kylin-X.Y.Z-incubating-rcN + +## Check in +$ cd ~/dist/dev/kylin +$ svn add apache-kylin-X.Y.Z-incubating-rcN +$ svn commit -m 'Upload release artifacts to staging' +{% endhighlight %} + +__Cleaning up after a failed release attempt:__ +{% highlight bash %} +# Make sure that the tag you are about to generate does not already +# exist (due to a failed release attempt) +$ git tag + +# If the tag exists, delete it locally and remotely +$ git tag -d apache-kylin-X.Y.Z-incubating +$ git push origin :refs/tags/apache-kylin-X.Y.Z-incubating + +# Remove modified files +$ mvn release:clean + +# Check whether there are modified files and if so, go back to the +# original git commit +$ git status +$ git reset --hard HEAD +{% endhighlight %} + +# Validate a release +{% highlight bash %} +# Check unit test +$ mvn test + +# Check that the signing key (e.g. 2AD3FAE3) is pushed +$ gpg --recv-keys key + +# Check keys +$ curl -O https://dist.apache.org/repos/dist/release/incubator/kylin/KEYS + +## Sign/check md5 and sha1 hashes + _(Assumes your O/S has 'md5' and 'sha1' commands.)_ +function checkHash() { + cd "$1" + for i in *.{zip,gz}; do + if [ ! -f $i ]; then + continue + fi + if [ -f $i.md5 ]; then + if [ "$(cat $i.md5)" = "$(md5 -q $i)" ]; then + echo $i.md5 present and correct + else + echo $i.md5 does not match + fi + else + md5 -q $i > $i.md5 + echo $i.md5 created + fi + if [ -f $i.sha1 ]; then + if [ "$(cat $i.sha1)" = "$(sha1 -q $i)" ]; then + echo $i.sha1 present and correct + else + echo $i.sha1 does not match + fi + else + sha1 -q $i > $i.sha1 + echo $i.sha1 created + fi + done +} +$ checkHash apache-kylin-X.Y.Z-incubating-rcN +{% endhighlight %} + +## Apache voting process + +__Vote on Apache Kylin dev mailing list__ +Release vote on dev list: + +{% highlight text %} +To: dev@kylin.incubator.apache.org +Subject: [VOTE] Release apache-kylin-X.Y.Z-incubating (release candidate N) + +Hi all, + +I have created a build for Apache Kylin X.Y.Z-incubating, release candidate N. + +Changes highlights: +... + +Thanks to everyone who has contributed to this release. +Here’s release notes: +https://github.com/apache/incubator-kylin/blob/XXX/docs/release_notes.md + +The commit to be voted upon: + +https://github.com/apache/incubator-kylin/commit/xxx + +Its hash is xxx. + +The artifacts to be voted on are located here: +https://dist.apache.org/repos/dist/dev/incubator/kylin/apache-kylin-X.Y.Z-incubating-rcN/ + +The hashes of the artifacts are as follows: +src.zip.md5 xxx +src.zip.sha1 xxx +src.tar.gz.md5 xxx +src.tar.gz.sha1 xxx + +A staged Maven repository is available for review at: +https://repository.apache.org/content/repositories/orgapachekylin-XXXX/ + +Release artifacts are signed with the following key: +https://people.apache.org/keys/committer/lukehan.asc + +Please vote on releasing this package as Apache Kylin X.Y.Z-incubating. + +The vote is open for the next 72 hours and passes if a majority of +at least three +1 PPMC votes are cast. + +[ ] +1 Release this package as Apache Kylin X.Y.Z-incubating +[ ] 0 I don't feel strongly about it, but I'm okay with the release +[ ] -1 Do not release this package because... + + +Here is my vote: + ++1 (binding) + +Luke + +{% endhighlight %} + +After vote finishes, send out the result: +{% highlight text %} +Subject: [RESULT] [VOTE] Release apache-kylin-X.Y.Z-incubating (release candidate N) +To: dev@kylin.incubator.apache.org + +Thanks to everyone who has tested the release candidate and given +their comments and votes. + +The tally is as follows. + +N binding +1s: + +N non-binding +1s: + +No 0s or -1s. + +Therefore I am delighted to announce that the proposal to release +Apache-Kylin-X.Y.Z-incubating has passed. + +I'll now start a vote on the general list. Those of you in the IPMC, +please recast your vote on the new thread. + +Luke + +{% endhighlight %} + +__Vote on Apache incubator general mailing list__ +Use the [Apache URL shortener](http://s.apache.org) to generate +shortened URLs for the vote proposal and result emails. Examples: +[http://s.apache.org/kylin-0.7.1-vote_rc3](http://s.apache.org/kylin-0.7.1-vote_rc3) and +[http://s.apache.org/kylin-0.7.1-result_rc3](http://s.apache.org/kylin-0.7.1-result_rc3). + +{% highlight text %} +To: general@incubator.apache.org +Subject: [VOTE] Release Apache Kylin X.Y.Z (incubating) + +Hi all, + +The Apache Kylin community has voted on and approved a proposal to release +Apache Kylin X.Y.Z (incubating). + +Proposal: +http://s.apache.org/kylin-X.Y.Z-vote_rcN + +Vote result: +N binding +1 votes +N non-binding +1 votes +No -1 votes +http://s.apache.org/kylin-X.Y.Z-result_rcN + + +The commit to be voted upon: +https://github.com/apache/incubator-kylin/commit/XXX + +Its hash is XXX. + +The artifacts to be voted on are located here: +https://dist.apache.org/repos/dist/dev/incubator/kylin/apache-kylin-X.Y.Z-incubating-rcN/ + +The hashes of the artifacts are as follows: +src.zip.md5 XXX +src.zip.sha1 XXX +src.tar.gz.md5 XXX +src.tar.gz.sha1 XXX + +A staged Maven repository is available for review at: +https://repository.apache.org/content/repositories/orgapachekylin-NNNN/ + +Release artifacts are signed with the following key: +https://people.apache.org/keys/committer/lukehan.asc + +Pursuant to the Releases section of the Incubation Policy and with +the endorsement of our mentors we would now like to request +the permission of the Incubator PMC to publish the release. The vote +is open for 72 hours, or until the necessary number of votes (3 +1) +is reached. + +[ ] +1 Release this package +[ ] 0 I don't feel strongly about it, but I'm okay with the release +[ ] -1 Do not release this package because... + + +Luke Han, on behalf of Apache Kylin PPMC +{% endhighlight %} + +After vote finishes, send out the result: +{% highlight text %} +To: general@incubator.apache.org +Subject: [RESULT] [VOTE] Release Apache Kylin X.Y.Z (incubating) + +This vote passes with N +1s and no 0 or -1 votes: + ++1 (binding or no-binding) + +Thanks everyone. We’ll now roll the release out to the mirrors. + +Luke Han, on behalf of Apache Kylin PPMC +{% endhighlight %} + +## Publishing a release +After a successful release vote, we need to push the release +out to mirrors, and other tasks. + +In JIRA, search for +[all issues resolved in this release](https://issues.apache.org/jira/issues/?jql=project%20%3D%20KYLIN%20), +and do a bulk update changing their status to "Closed", +with a change comment +"Resolved in release X.Y.Z-incubating (YYYY-MM-DD)" +(fill in release number and date appropriately). +__Uncheck "Send mail for this update".__ + +Promote the staged nexus artifacts. + +* Go to [https://repository.apache.org/](https://repository.apache.org/) and login +* Under "Build Promotion" click "Staging Repositories" +* In the line with "orgapachekylin-xxxx", check the box +* Press "Release" button + +Check the artifacts into svn. +{% highlight bash %} +# Get the release candidate. +$ mkdir -p ~/dist/dev +$ cd ~/dist/dev +$ svn co https://dist.apache.org/repos/dist/dev/incubator/kylin + +# Copy the artifacts. Note that the copy does not have '-rcN' suffix. +$ mkdir -p ~/dist/release +$ cd ~/dist/release +$ svn co https://dist.apache.org/repos/dist/release/incubator/kylin +$ cd kylin +$ cp -rp ../../dev/kylin/apache-kylin-X.Y.Z-incubating-rcN apache-kylin-X.Y.Z-incubating +$ svn add apache-kylin-X.Y.Z-incubating + +# Check in. +svn commit -m 'checkin release artifacts' +{% endhighlight %} + +Svnpubsub will publish to +[https://dist.apache.org/repos/dist/release/incubator/kylin](https://dist.apache.org/repos/dist/release/incubator/kylin) and propagate to +[http://www.apache.org/dyn/closer.cgi/incubator/kylin](http://www.apache.org/dyn/closer.cgi/incubator/kylin) within 24 hours. + +If there are now more than 2 releases, clear out the oldest ones: + +{% highlight bash %} +cd ~/dist/release/kylin +svn rm apache-kylin-X.Y.Z-incubating +svn commit -m 'Remove old release' +{% endhighlight %} + +The old releases will remain available in the +[release archive](http://archive.apache.org/dist/incubator/kylin/). + +Release same version in JIRA, check [Change Log](https://issues.apache.org/jira/browse/KYLIN/?selectedTab=com.atlassian.jira.jira-projects-plugin:changelog-panel) for the latest released version. + +## Publishing the web site +Refer to [How to document](howto_docs.html) for more detail. + +# Thanks +This guide drafted with reference from [Apache Calcite](http://calcite.incubator.apache.org) Howto doc, Thank you very much. + http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_dev/javascript_api.md ---------------------------------------------------------------------- diff --git a/website/_dev/javascript_api.md b/website/_dev/javascript_api.md deleted file mode 100644 index fc76e4d..0000000 --- a/website/_dev/javascript_api.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -layout: dev -title: How to consume Kylin REST API in javascript -categories: development -permalink: /development/javascript_api.html ---- -Kypin securicty authorization is based on basic access authorization, so when you want to use API in your javascript,you need to involve the authorization info in http headers. - -## Here is a demo show how to use Kylin Query API. -``` -$.ajaxSetup({ - headers: { 'Authorization': "Basic eWFu**********X***ZA==", 'Content-Type': 'application/json;charset=utf-8' } // use your own authorization code here - }); - var request = $.ajax({ - url: "http://hostname/kylin/api/query", - type: "POST", - data: '{"sql":"select count(*) from SUMMARY;","offset":0,"limit":50000,"partial":true,"project":"test"}', - dataType: "json" - }); - request.done(function( msg ) { - alert(msg); - }); - request.fail(function( jqXHR, textStatus ) { - alert( "Request failed: " + textStatus ); - }); - -``` - - -For what is basic access authorization ,click here http://en.wikipedia.org/wiki/Basic_access_authentication - -## Keypoint: -1. add basic access authorization info in heders. -2. use right ajax type and right data synax - -How to generate your authorization code (download and import "jquery.base64.js" from https://github.com/yckart/jquery.base64.js) - -``` -var authorizationCode = $.base64('encode', 'NT_USERNAME' + ":" + 'NT_PASSWORD'); - -$.ajaxSetup({ - headers: { - 'Authorization': "Basic " + authorizationCode, - 'Content-Type': 'application/json;charset=utf-8' - } -}); -``` http://git-wip-us.apache.org/repos/asf/incubator-kylin/blob/6d59b249/website/_dev/metadata_store.md ---------------------------------------------------------------------- diff --git a/website/_dev/metadata_store.md b/website/_dev/metadata_store.md deleted file mode 100644 index 199a40a..0000000 --- a/website/_dev/metadata_store.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -layout: dev -title: "Kylin Metadata Store" -categories: development -permalink: /development/metadata_store.html ---- - -Kylin organizes all of its metadata(including cube descriptions and instances, projects, inverted index description and instances, jobs, tables and dictionaries) as a hierarchy file system. However, Kylin uses hbase to store it, rather than normal file system. If you check your kylin configuration file(kylin.properties) you will find such a line: - -{% highlight Groff markup %} -## The metadata store in hbase -kylin.metadata.url=kylin_metadata_qa@hbase:sandbox.hortonworks.com:2181:/hbase-unsecure -{% endhighlight %} - - -This indicates that the metadata will be saved as a htable called `kylin_metadata_qa`. You can scan the htable in hbase shell to check it out. - -## Backup Metadata Store with binary package - -Sometimes you need to backup the Kylin's Metadata Store from hbase to your disk file system. -In such cases, assuming you're on the hadoop CLI(or sandbox) where you deployed Kylin, you can go to KYLIN_HOME and run : - -{% highlight Groff markup %} -bin/metastore.sh backup -{% endhighlight %} - -to dump your metadata to your local folder a folder under KYLIN_HOME/metadata_backps, the folder is named after current time with the syntax: KYLIN_HOME/meta_backups/meta_year_month_day_hour_minute_second - -## Restore Metadata Store with binary package - -In case you find your metadata store messed up, and you want to restore to a previous backup: - -Firstly, reset the metadata store (this will clean everything of the Kylin metadata store in hbase, make sure to backup): - -{% highlight Groff markup %} -bin/metastore.sh reset -{% endhighlight %} - -Then upload the backup metadata to Kylin's metadata store: -{% highlight Groff markup %} -bin/metastore.sh restore $KYLIN_HOME/meta_backups/meta_xxxx_xx_xx_xx_xx_xx -{% endhighlight %} - -## Backup/Restore metadata in development env (available since 0.7.3) - -When developing/debugging Kylin, typically you have a dev machine with an IDE, and a backend sandbox. Usually you'll write code and run test cases at dev machine. It would be troublesome if you always have to put a binary package in the sandbox to check the metadata. There is a helper class called SandboxMetastoreCLI to help you download/upload metadata locally at your dev machine. Follow the Usage information and run it in your IDE. - -## Cleanup unused resources from Metadata Store (available since 0.7.3) -As time goes on, some resources like dictionary, table snapshots became useless (as the cube segment be dropped or merged), but they still take space there; You can run command to find and cleanup them from metadata store: - -Firstly, run a check, this is safe as it will not change anything: -{% highlight Groff markup %} -bin/metastore.sh clean -{% endhighlight %} - -The resources that will be dropped will be listed; - -Next, add the "--delete true" parameter to cleanup those resources; before this, make sure you have made a backup of the metadata store; -{% highlight Groff markup %} -bin/metastore.sh clean --delete true -{% endhighlight %}