Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 588DA2009C6 for ; Tue, 31 May 2016 12:29:39 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 5715E160A43; Tue, 31 May 2016 10:29:39 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 76F0D160A01 for ; Tue, 31 May 2016 12:29:38 +0200 (CEST) Received: (qmail 14104 invoked by uid 500); 31 May 2016 10:29:37 -0000 Mailing-List: contact commits-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list commits@flink.apache.org Received: (qmail 14095 invoked by uid 99); 31 May 2016 10:29:37 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 May 2016 10:29:37 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 95BE3DFB74; Tue, 31 May 2016 10:29:37 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: mxm@apache.org To: commits@flink.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: flink git commit: [FLINK-3887] improve dependency management for building docs Date: Tue, 31 May 2016 10:29:37 +0000 (UTC) archived-at: Tue, 31 May 2016 10:29:39 -0000 Repository: flink Updated Branches: refs/heads/release-0.10 9f80376a3 -> 644c27504 [FLINK-3887] improve dependency management for building docs The Flink documentation build process is currently quite messy. These changes move us to a new build process with proper dependency handling. It assures that we all use the same dependency versions for consistent build output. Also, it eases the automated building process on other systems (like the ASF Buildbot). The goal was to make the documentation build process easier and self-contained. - use Ruby's Bundler Gem to install dependencies - update README - adapt Dockerfile - add additional rules to .gitignore - change default doc output path from /target to /content (default path of the flink-web repository) This closes #2033 Project: http://git-wip-us.apache.org/repos/asf/flink/repo Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/644c2750 Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/644c2750 Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/644c2750 Branch: refs/heads/release-0.10 Commit: 644c27504ad6fb89372e3b39123a4f896013e1ad Parents: 9f80376 Author: Maximilian Michels Authored: Mon May 16 15:02:59 2016 +0200 Committer: Maximilian Michels Committed: Tue May 31 12:27:23 2016 +0200 ---------------------------------------------------------------------- .gitignore | 7 +++++-- docs/Gemfile | 25 +++++++++++++++++++++++++ docs/README.md | 26 +++++++++++++++----------- docs/build_docs.sh | 42 ++++++++++++++++++------------------------ pom.xml | 5 +++-- 5 files changed, 66 insertions(+), 39 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flink/blob/644c2750/.gitignore ---------------------------------------------------------------------- diff --git a/.gitignore b/.gitignore index 638f71f..10565ac 100644 --- a/.gitignore +++ b/.gitignore @@ -16,10 +16,13 @@ tmp *.jar *.log .DS_Store -_site -docs/api build-target flink-staging/flink-avro/src/test/java/org/apache/flink/api/io/avro/generated/ flink-runtime-web/web-dashboard/node_modules/ flink-runtime-web/web-dashboard/bower_components/ atlassian-ide-plugin.xml +/docs/api +/docs/content +/docs/Gemfile.lock +/docs/.bundle +/docs/.rubydeps http://git-wip-us.apache.org/repos/asf/flink/blob/644c2750/docs/Gemfile ---------------------------------------------------------------------- diff --git a/docs/Gemfile b/docs/Gemfile new file mode 100644 index 0000000..d367999 --- /dev/null +++ b/docs/Gemfile @@ -0,0 +1,25 @@ +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +source 'https://rubygems.org' + +# Dependencies required to build the Flink docs +gem 'jekyll', '2.5.3' +gem 'kramdown', '1.10.0' +gem 'pygments.rb', '0.6.3' +gem 'therubyracer', '0.12.2' http://git-wip-us.apache.org/repos/asf/flink/blob/644c2750/docs/README.md ---------------------------------------------------------------------- diff --git a/docs/README.md b/docs/README.md index 206b04c..a9f6e28 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,23 +6,27 @@ http://flink.apache.org/ is also generated from the files found here. # Requirements -We use Markdown to write and Jekyll to translate the documentation to static HTML. You can install -all needed software via: +The dependencies are declared in the Gemfile in this directory. We use Markdown +to write and Jekyll to translate the documentation to static HTML. All required +dependencies are installed locally when you build the documentation through the +`build_docs.sh` script. If you want to install the software manually, use Ruby's +Bundler Gem to install all dependencies: - gem install jekyll - gem install kramdown - sudo easy_install Pygments + gem install bundler + bundle install -Kramdown is needed for Markdown processing and the Python based Pygments is used for syntax -highlighting. +Note that in Ubuntu based systems, it may be necessary to install the `ruby-dev` +via apt to build native code. # Build -The `docs/build_docs.sh` script calls Jekyll and generates the documentation in `docs/target`. You -can then point your browser to `docs/target/index.html` and start reading. +The `docs/build_docs.sh` script installs dependencies locally, calls Jekyll, and +generates the documentation in `docs/content`. You can then point your browser +to `docs/content/index.html` and start reading. -If you call the script with the preview flag `build_docs.sh -p`, Jekyll will start a web server at -`localhost:4000` and watch the docs directory for updates. Use this mode to preview changes locally. +If you call the script with the preview flag `build_docs.sh -p`, Jekyll will +start a web server at `localhost:4000` and watch the docs directory for +updates. Use this mode to preview changes locally. # Contribute http://git-wip-us.apache.org/repos/asf/flink/blob/644c2750/docs/build_docs.sh ---------------------------------------------------------------------- diff --git a/docs/build_docs.sh b/docs/build_docs.sh index b65f7c9..17561c2 100755 --- a/docs/build_docs.sh +++ b/docs/build_docs.sh @@ -17,35 +17,30 @@ # limitations under the License. ################################################################################ +set -e +cd "$(dirname ${BASH_SOURCE[0]})" -HAS_JEKYLL=true +DIR="`pwd`" -command -v jekyll > /dev/null -if [ $? -ne 0 ]; then - echo -n "ERROR: Could not find jekyll. " - echo "Please install with 'gem install jekyll' (see http://jekyllrb.com)." +# We need at least bundler to proceed +if [ "`command -v bundle`" == "" ]; then + echo "WARN: Could not find bundle." + echo "Attempting to install locally. If this doesn't work, please install with 'gem install bundler'." - HAS_JEKYLL=false -fi - -command -v redcarpet > /dev/null -if [ $? -ne 0 ]; then - echo -n "WARN: Could not find redcarpet. " - echo -n "Please install with 'sudo gem install redcarpet' (see https://github.com/vmg/redcarpet). " - echo "Redcarpet is needed for Markdown parsing and table of contents generation." -fi + # Adjust the PATH to discover the locally installed Ruby gem + if which ruby >/dev/null && which gem >/dev/null; then + export PATH="$(ruby -rubygems -e 'puts Gem.user_dir')/bin:$PATH" + fi -command -v pygmentize > /dev/null -if [ $? -ne 0 ]; then - echo -n "WARN: Could not find pygments. " - echo -n "Please install with 'sudo easy_install Pygments' (requires Python; see http://pygments.org). " - echo "Pygments is needed for syntax highlighting of the code examples." + # install bundler locally + gem install --user-install bundler fi -DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# Install Ruby dependencies locally +bundle install --path .rubydeps DOCS_SRC=${DIR} -DOCS_DST=${DOCS_SRC}/target +DOCS_DST=${DOCS_SRC}/content # default jekyll command is to just build site JEKYLL_CMD="build" @@ -59,6 +54,5 @@ while getopts ":p" opt; do esac done -if $HAS_JEKYLL; then - jekyll ${JEKYLL_CMD} --source ${DOCS_SRC} --destination ${DOCS_DST} -fi +# use 'bundle exec' to insert the local Ruby dependencies +bundle exec jekyll ${JEKYLL_CMD} --source "${DOCS_SRC}" --destination "${DOCS_DST}" http://git-wip-us.apache.org/repos/asf/flink/blob/644c2750/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 99bb191..28570ea 100644 --- a/pom.xml +++ b/pom.xml @@ -772,11 +772,12 @@ under the License. - **/.* + **/.*/** **/*.prefs **/*.log docs/**/bootstrap* + docs/Gemfile.lock docs/img/*.svg **/resources/**/font-awesome/** **/resources/**/jquery* @@ -826,7 +827,7 @@ under the License. flink-quickstart/**/testArtifact/goal.txt **/target/** - docs/_site/** + docs/content/** **/scalastyle-output.xml build-target/**