superset-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maximebeauche...@apache.org
Subject [incubator-superset] branch master updated: Clean up CONTRIBUTING.md: (#5911)
Date Wed, 19 Sep 2018 17:01:57 GMT
This is an automated email from the ASF dual-hosted git repository.

maximebeauchemin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git


The following commit(s) were added to refs/heads/master by this push:
     new 549328f  Clean up CONTRIBUTING.md: (#5911)
549328f is described below

commit 549328f8f08949809355010e347c31a1dd7ae341
Author: Alek Storm <alek.storm@gmail.com>
AuthorDate: Wed Sep 19 12:01:51 2018 -0500

    Clean up CONTRIBUTING.md: (#5911)
    
    - Reorganize sections for better navigability
    - Add table of contents
    - Rework frontend assets section for clarity and DRY
    - Rework translating section for clarity, add "Add Translations" contribution type
    - Move release docs only useful for maintainers to RELEASING.md
    - Other miscellaneous improvements
---
 CONTRIBUTING.md                        | 655 +++++++++++++++------------------
 RELEASING.md                           |  93 +++++
 docs/installation.rst                  |  14 +-
 superset/translations/requirements.txt |   1 +
 4 files changed, 388 insertions(+), 375 deletions(-)

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index e1357ee..b4d325b 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -3,59 +3,80 @@
 Contributions are welcome and are greatly appreciated! Every
 little bit helps, and credit will always be given.
 
-You can contribute in many ways:
+## Table of Contents
+
+- [Types of Contributions](#types-of-contributions)
+  - [Report Bugs](#report-bugs)
+  - [Fix Bugs](#fix-bugs)
+  - [Implement Features](#implement-features)
+  - [Improve Documentation](#improve-documentation)
+  - [Add Translations](#add-translations)
+  - [Submit Feedback](#submit-feedback)
+  - [Ask Questions](#ask-questions)
+- [Pull Request Guidelines](#pull-request-guidelines)
+- [Local development](#local-development)
+  - [Documentation](#documentation)
+  - [Flask server](#flask-server)
+  - [Frontend assets](#frontend-assets)
+- [Testing](#testing)
+  - [JavaScript testing](#javascript-testing)
+  - [Integration testing](#integration-testing)
+  - [Linting](#linting)
+- [Translating](#translating)
+  - [Enabling language selection](#enabling-language-selection)
+  - [Extracting new strings for translation](#extracting-new-strings-for-translation)
+  - [Creating a new language dictionary](#creating-a-new-language-dictionary)
+- [Tips](#tips)
+  - [Adding a new datasource](#adding-a-new-datasource)
+  - [Creating a new visualization type](#creating-a-new-visualization-type)
+  - [Adding a DB migration](#adding-a-db-migration)
+  - [Merging DB migrations](#merging-db-migrations)
 
 ## Types of Contributions
 
 ### Report Bugs
 
-Report bugs through GitHub
+Report bugs through GitHub. If you are reporting a bug, please include:
 
-If you are reporting a bug, please include:
+- Your operating system name and version.
+- Any details about your local setup that might be helpful in troubleshooting.
+- Detailed steps to reproduce the bug.
 
--   Your operating system name and version.
--   Any details about your local setup that might be helpful in
-    troubleshooting.
--   Detailed steps to reproduce the bug.
-
-When you post python stack traces please quote them using
-[markdown blocks](https://help.github.com/articles/creating-and-highlighting-code-blocks/).
+When posting Python stack traces, please quote them using
+[Markdown blocks](https://help.github.com/articles/creating-and-highlighting-code-blocks/).
 
 ### Fix Bugs
 
-Look through the GitHub issues for bugs. Anything tagged with "bug" is
+Look through the GitHub issues for bugs. Anything tagged with `bug` is
 open to whoever wants to implement it.
 
 ### Implement Features
 
 Look through the GitHub issues for features. Anything tagged with
-"feature" or "starter_task" is open to whoever wants to implement it.
+`feature` or `starter_task` is open to whoever wants to implement it.
 
-### Documentation
+### Improve Documentation
 
 Superset could always use better documentation,
 whether as part of the official Superset docs,
 in docstrings, `docs/*.rst` or even on the web as blog posts or
-articles.
+articles. See [Documentation](#documentation) for more details.
 
-To build the docs, simply run ``./build.sh`` from the ``docs/`` folder.
-The rendered docs are accessible at `http://localhost:{PORT}/static/assets/docs/faq.html`.
+### Add Translations
 
-### Submit Feedback
+If you are proficient in a non-English language, you can help translate text strings from
Superset's UI. You can jump in to the existing language dictionaries at `superset/translations/<language_code>/LC_MESSAGES/messages.po`,
or even create a dictionary for a new language altogether. See [Translating](#translating)
for more details.
 
-The best way to send feedback is to file an issue on GitHub.
+### Submit Feedback
 
-If you are proposing a feature:
+The best way to send feedback is to file an issue on GitHub. If you are proposing a feature:
 
--   Explain in detail how it would work.
--   Keep the scope as narrow as possible, to make it easier to
-    implement.
--   Remember that this is a volunteer-driven project, and that
-    contributions are welcome :)
+- Explain in detail how it would work.
+- Keep the scope as narrow as possible, to make it easier to implement.
+- Remember that this is a volunteer-driven project, and that contributions are welcome :)
 
-### Questions
+### Ask Questions
 
-There is a dedicated [tag](https://stackoverflow.com/questions/tagged/apache-superset) on
[stackoverflow](https://stackoverflow.com/). Please use it when asking questions.
+There is a dedicated [`apache-superset` tag](https://stackoverflow.com/questions/tagged/apache-superset)
on [StackOverflow](https://stackoverflow.com/). Please use it when asking questions.
 
 ## Pull Request Guidelines
 
@@ -79,9 +100,18 @@ meets these guidelines:
 8.  If you are asked to update your pull request with some changes there's
     no need to create a new one. Push your changes to the same branch.
 
-## Documentation
+## Local development
 
-The latest documentation and tutorial are available [here](https://superset.incubator.apache.org/).
+First, [fork the repository on GitHub](https://help.github.com/articles/about-forks/), then
clone it. You can clone the main repository directly instead, but you won't be able to send
pull requests.
+
+```bash
+git clone git@github.com:your-username/incubator-superset.git
+cd incubator-superset
+```
+
+### Documentation
+
+The latest documentation and tutorial are available at https://superset.incubator.apache.org/.
 
 Contributing to the official documentation is relatively easy, once you've setup
 your environment and done an edit end-to-end. The docs can be found in the
@@ -92,514 +122,411 @@ If you've written Markdown before, you'll find the reStructuredText
format famil
 Superset uses [Sphinx](http://www.sphinx-doc.org/en/1.5.1/) to convert the rst files
 in `docs/` to the final HTML output users see.
 
-Before you start changing the docs, you'll want to
-[fork the Superset project on Github](https://help.github.com/articles/fork-a-repo/).
-Once that new repository has been created, clone it on your local machine:
-
-    git clone git@github.com:your_username/incubator-superset.git
-
-At this point, you may also want to create a
-[Python virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/)
-to manage the Python packages you're about to install:
-
-    virtualenv superset-dev
-    source superset-dev/bin/activate
-
 Finally, to make changes to the rst files and build the docs using Sphinx,
 you'll need to install a handful of dependencies from the repo you cloned:
 
-    cd incubator-superset
-    pip install -r docs/requirements.txt
+```bash
+pip install -r docs/requirements.txt
+```
 
 To get the feel for how to edit and build the docs, let's edit a file, build
 the docs and see our changes in action. First, you'll want to
 [create a new branch](https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging)
 to work on your changes:
 
-    git checkout -b changes-to-docs
+```bash
+git checkout -b changes-to-docs
+```
 
-Now, go ahead and edit one of the files under `docs/`, say `docs/tutorial.rst`
-- change it however you want. Check out the
+Now, go ahead and edit one of the files under `docs/`, say `docs/tutorial.rst` - change
+it however you want. Check out the
 [ReStructuredText Primer](http://docutils.sourceforge.net/docs/user/rst/quickstart.html)
 for a reference on the formatting of the rst files.
 
-Once you've made your changes, run this command from the root of the Superset
-repo to convert the docs into HTML:
+Once you've made your changes, run this command to convert the docs into HTML:
 
-    python setup.py build_sphinx
+```bash
+make html
+```
 
 You'll see a lot of output as Sphinx handles the conversion. After it's done, the
-HTML Sphinx generated should be in `docs/_build/html`. Go ahead and navigate there
+HTML Sphinx generated should be in `docs/_build/html`. Navigate there
 and start a simple web server so we can check out the docs in a browser:
 
-    cd docs/_build/html
-    python -m SimpleHTTPServer
+```bash
+cd docs/_build/html
+python -m SimpleHTTPServer
+```
 
 This will start a small Python web server listening on port 8000. Point your
-browser to [http://localhost:8000/](http://localhost:8000/), find the file
+browser to http://localhost:8000, find the file
 you edited earlier, and check out your changes!
 
 If you've made a change you'd like to contribute to the actual docs, just commit
 your code, push your new branch to Github:
 
-    git add docs/tutorial.rst
-    git commit -m 'Awesome new change to tutorial'
-    git push origin changes-to-docs
+```bash
+git add docs/tutorial.rst
+git commit -m 'Awesome new change to tutorial'
+git push origin changes-to-docs
+```
 
 Then, [open a pull request](https://help.github.com/articles/about-pull-requests/).
 
+#### Images
+
 If you're adding new images to the documentation, you'll notice that the images
 referenced in the rst, e.g.
 
     .. image:: _static/img/tutorial/tutorial_01_sources_database.png
 
-aren't actually included in that directory. _Instead_, you'll want to add and commit
-images (and any other static assets) to the _superset/assets/images_ directory.
-When the docs are being pushed to [Apache Superset (incubating)](https://superset.incubator.apache.org/),
images
-will be moved from there to the _\_static/img_ directory, just like they're referenced
+aren't actually stored in that directory. Instead, you should add and commit
+images (and any other static assets) to the `superset/assets/images` directory.
+When the docs are deployed to https://superset.incubator.apache.org/, images
+are copied from there to the `_static/img` directory, just like they're referenced
 in the docs.
 
-For example, the image referenced above actually lives in
-
-    superset/assets/images/tutorial
-
-Since the image is moved during the documentation build process, the docs reference the
-image in
+For example, the image referenced above actually lives in `superset/assets/images/tutorial`.
Since the image is moved during the documentation build process, the docs reference the image
in `_static/img/tutorial` instead.
 
-    _static/img/tutorial
+#### API documentation
 
-instead.
+Generate the API documentation with:
 
-## Setting up a Python development environment
-
-Check the [OS dependencies](https://superset.incubator.apache.org/installation.html#os-dependencies)
before follows these steps.
+```bash
+pip install -r docs/requirements.txt
+python setup.py build_sphinx
+```
 
-    # fork the repo on GitHub and then clone it
-    # alternatively you may want to clone the main repo but that won't work
-    # so well if you are planning on sending PRs
-    # git clone git@github.com:apache/incubator-superset.git
+### Flask server
 
-    # [optional] setup a virtual env and activate it
-    virtualenv env
-    source env/bin/activate
+Make sure your machine meets the [OS dependencies](https://superset.incubator.apache.org/installation.html#os-dependencies)
before following these steps.
 
-    # install for development
-    pip install -r requirements.txt
-    pip install -e .
+```bash
+# Create a virtual environemnt and activate it (recommended)
+virtualenv venv
+source venv/bin/activate
 
-    # Create an admin user
-    fabmanager create-admin --app superset
+# Install external dependencies
+pip install -r requirements.txt
+# Install Superset in editable (development) mode
+pip install -e .
 
-    # Initialize the database
-    superset db upgrade
+# Create an admin user
+fabmanager create-admin --app superset
 
-    # Create default roles and permissions
-    superset init
+# Initialize the database
+superset db upgrade
 
-    # Load some data to play with
-    superset load_examples
+# Create default roles and permissions
+superset init
 
-    # start a dev web server
-    superset runserver -d
+# Load some data to play with
+superset load_examples
 
+# Start the Flask web server (but see below for frontend asset compilation)
+superset runserver -d
+```
 
-### Logging to the browser console (Python 3 only)
+#### Logging to the browser console
 
-When debugging your application, you can have the server logs sent directly to the browser
console:
+This feature is only available on Python 3. When debugging your application, you can have
the server logs sent directly to the browser console:
 
-    superset runserver -d --console-log
+```bash
+superset runserver -d --console-log
+```
 
 You can log anything to the browser console, including objects:
 
-    from superset import app
-    app.logger.error('An exception occurred!')
-    app.logger.info(form_data)
-
-
-## Setting up the node / npm javascript environment
-
-`superset/assets` contains all npm-managed, front end assets.
-Flask-Appbuilder itself comes bundled with jQuery and bootstrap.
-While these may be phased out over time, these packages are currently not
-managed with npm.
+```python
+from superset import app
+app.logger.error('An exception occurred!')
+app.logger.info(form_data)
+```
 
-### Node/npm versions
-Make sure you are using recent versions of node and npm. No problems have been found with
node>=5.10 and 4.0. > npm>=3.9.
+### Frontend assets
 
-### Using npm to generate bundled files
+Frontend assets (JavaScript, CSS, and images) must be compiled in order to properly display
the web UI. The `superset/assets` directory contains all NPM-managed front end assets. Note
that there are additional frontend assets bundled with Flask-Appbuilder (e.g. jQuery and bootstrap);
these are not managed by NPM, and may be phased out in the future.
 
-#### npm
-First, npm must be available in your environment. If it is not you can run the following
commands
-(taken from [this source](https://gist.github.com/DanHerbert/9520689))
-```
-brew install node --without-npm
-echo prefix=~/.npm-packages >> ~/.npmrc
-curl -L https://www.npmjs.com/install.sh | sh
-```
-
-The final step is to add `~/.npm-packages/bin` to your `PATH` so commands you install globally
are usable.
-Add something like this to your `.bashrc` file, then `source ~/.bashrc` to reflect the change.
-```
-export PATH="$HOME/.npm-packages/bin:$PATH"
-```
+First, be sure you are using recent versions of NodeJS and npm. Using [nvm](https://github.com/creationix/nvm)
to manage them is recommended.
 
-#### npm packages
-To install third party libraries defined in `package.json`, run the
-following within the `superset/assets/` directory which will install them in a
-new `node_modules/` folder within `assets/`.
+Install third-party dependencies listed in `package.json`:
 
 ```bash
-# from the root of the repository, move to where our JS package.json lives
-cd superset/assets/
-# install yarn, a replacement for `npm install` that is faster and more deterministic
+# From the root of the repository
+cd superset/assets
+
+# Install yarn, a replacement for `npm install`
 npm install -g yarn
-# run yarn to fetch all the dependencies
-yarn
+
+# Install dependencies
+yarn install
 ```
 
-To parse and generate bundled files for superset, run either of the
-following commands. The `dev` flag will keep the npm script running and
-re-run it upon any changes within the assets directory.
+Finally, to compile frontend assets, run any of the following commands.
 
 ```bash
-# Copies a conf file from the frontend to the backend
-npm run sync-backend
+# Start a watcher that recompiles your assets as you modify them (reload your browser to
see changes)
+npm run dev
 
-# Compiles the production / optimized js & css
+# Compile the Javascript and CSS in production/optimized mode for official releases
 npm run prod
 
-# Start a watcher that rebundle your assets as you modify them
-npm run dev
-
-# Start a web server that manages and updates your assets as you modify them
-npm run dev-server
+# Copy a conf file from the frontend to the backend
+npm run sync-backend
 ```
 
-For every development session you will have to
+#### Webpack dev server
 
-1. Start a flask dev server
-
-```bash
-superset runserver -d
-# or specify port
-superset runserver -d -p 8081
-```
-
-2. Start webpack dev server
+Alternatively, you can run the Webpack dev server, which runs on port 9000 and proxies non-asset
requests to the Flask server on port 8088. After pointing your browser to it, updates to asset
sources will be reflected in-browser without a refresh.
 
 ```bash
+# Run the dev server
 npm run dev-server
-```
-
-This will start `webpack-dev-server` at port 9000 and you can access Superset at localhost:9000.
-By default, `webpack-dev-server` is configured for flask running at port 8088.
 
-If you start flask server at another port (e.g. 8081), you have to pass an extra argument
-`supersetPort` to `webpack-dev-server`
+# Run the dev server on a non-default port
+npm run dev-server -- --port=9001
 
-```bash
+# Run the dev server proxying to a Flask server on a non-default port
 npm run dev-server -- --supersetPort=8081
 ```
 
-You can also specify port for `webpack-dev-server`
-
-```bash
-npm run dev-server -- --port=9001
-# or with both dev-server port and superset port
-npm run dev-server -- --port=9001 --supersetPort=8081
-```
-
-#### Upgrading npm packages
+#### Upgrading NPM packages
 
-Should you add or upgrade a npm package, which involves changing `package.json`, you'll need
to re-run `yarn install` and push the newly generated `yarn.lock` file so we get the reproducible
build. More information at (https://yarnpkg.com/blog/2016/11/24/lockfiles-for-all/)
+After adding or upgrading an NPM package by changing `package.json`, you must run `yarn install`,
which will regenerate the `yarn.lock` file. Then, be sure to commit the new `yarn.lock` so
that other users' builds are reproducible. See [the Yarn docs](https://yarnpkg.com/blog/2016/11/24/lockfiles-for-all/)
for more information.
 
 ## Testing
+
 All tests are carried out in [tox](http://tox.readthedocs.io/en/latest/index.html)
 a standardized testing framework mostly for Python (though we also used it for Javascript).
 All python tests can be run with any of the tox [environments](http://tox.readthedocs.io/en/latest/example/basic.html#a-simple-tox-ini-default-environments),
via,
 
-    tox -e <environment>
+```bash
+tox -e <environment>
+```
 
 i.e.,
 
-    tox -e py27
-    tox -e py36
+```bash
+tox -e py27
+tox -e py36
+```
 
 Alternatively, you can run all tests in a single file via,
 
-    tox -e <environment> -- tests/test_file.py
+```bash
+tox -e <environment> -- tests/test_file.py
+```
 
 or for a specific test via,
 
-    tox -e <environment> -- tests/test_file.py:TestClassName.test_method_name
+```bash
+tox -e <environment> -- tests/test_file.py:TestClassName.test_method_name
+```
 
 Note that the test environment uses a temporary directory for defining the
 SQLite databases which will be cleared each time before the group of test
 commands are invoked.
 
+### JavaScript testing
+
 We use [Mocha](https://mochajs.org/), [Chai](http://chaijs.com/) and [Enzyme](http://airbnb.io/enzyme/)
to test Javascript. Tests can be run with:
 
-    cd /superset/superset/assets/javascripts
-    npm i
-    npm run test
+```bash
+cd superset/assets/javascripts
+npm install
+npm run test
+```
+
+### Integration testing
 
 We use [Cypress](https://www.cypress.io/) for integration tests. Tests can be run by `tox
-e cypress`. To open Cypress and explore tests first setup and run test server:
 
-    export SUPERSET_CONFIG=tests.superset_test_config
-    superset db upgrade
-    superset init
-    superset load_test_users
-    superset load_examples
-    superset runserver
+```bash
+export SUPERSET_CONFIG=tests.superset_test_config
+superset db upgrade
+superset init
+superset load_test_users
+superset load_examples
+superset runserver
+```
 
-Open Cypress tests:
+Run Cypress tests:
 
-    cd /superset/superset/assets
-    npm run build
-    npm run cypress run
+```bash
+cd /superset/superset/assets
+npm run build
+npm run cypress run
+```
 
-## Linting
+### Linting
 
 Lint the project with:
 
-    # for python
-    tox -e flake8
-
-    # for javascript
-    tox -e eslint
-
-## API documentation
-
-Generate the documentation with:
-
-    pip install -r docs/requirements.txt
-    python setup.py build_sphinx
+```bash
+# for python
+tox -e flake8
 
-## CSS Themes
-As part of the npm build process, CSS for Superset is compiled from `Less`, a dynamic stylesheet
language.
+# for javascript
+tox -e eslint
+```
 
-It's possible to customize or add your own theme to Superset, either by overriding CSS rules
or preferably
-by modifying the Less variables or files in `assets/stylesheets/less/`.
+## Translating
 
-The `variables.less` and `bootswatch.less` files that ship with Superset are derived from
-[Bootswatch](https://bootswatch.com) and thus extend Bootstrap. Modify variables in these
files directly, or
-swap them out entirely with the equivalent files from other Bootswatch (themes)[https://github.com/thomaspark/bootswatch.git]
+We use [Babel](http://babel.pocoo.org/en/latest/) to translate Superset. In Python files,
we import the magic `_` function using:
 
-## Translations
+```python
+from flask_babel import lazy_gettext as _
+```
 
-We use [Babel](http://babel.pocoo.org/en/latest/) to translate Superset. The
-key is to instrument the strings that need translation using
-`from flask_babel import lazy_gettext as _`. Once this is imported in
-a module, all you have to do is to `_("Wrap your strings")` using the
-underscore `_` "function".
+then wrap our translatable strings with it, e.g. `_('Translate me')`. During extraction,
string literals passed to `_` will be added to the generated `.po` file for each language
for later translation.
+At runtime, the `_` function will return the translation of the given string for the current
language, or the given string itself if no translation is available.
 
-We use `import {t, tn, TCT} from locales;` in js, JSX file, locales is in `./superset/assets/javascripts/`
directory.
+In JavaScript, the technique is similar: we import `t` (simple translation), `tn` (translation
containing a number), and `TCT` (translating entire React Components).
 
-To enable changing language in your environment, you can simply add the
-`LANGUAGES` parameter to your `superset_config.py`. Having more than one
-options here will add a language selection dropdown on the right side of the
-navigation bar.
+```javascript
+import {t, tn, TCT} from locales;
+```
 
-    LANGUAGES = {
-        'en': {'flag': 'us', 'name': 'English'},
-        'fr': {'flag': 'fr', 'name': 'French'},
-        'zh': {'flag': 'cn', 'name': 'Chinese'},
-    }
+### Enabling language selection
 
-As per the [Flask AppBuilder documentation] about translation, to create a
-new language dictionary, run the following command (where `es` is replaced with
-the language code for your target language):
+Add the `LANGUAGES` variable to your `superset_config.py`. Having more than one
+option inside will add a language selection dropdown to the UI on the right side
+of the navigation bar.
 
-    pybabel init -i superset/translations/messages.pot -d superset/translations -l es
+```python
+LANGUAGES = {
+    'en': {'flag': 'us', 'name': 'English'},
+    'fr': {'flag': 'fr', 'name': 'French'},
+    'zh': {'flag': 'cn', 'name': 'Chinese'},
+}
+```
 
-Then it's a matter of running the statement below to gather all strings that
-need translation
+### Extracting new strings for translation
 
-    fabmanager babel-extract --target superset/translations/ --output superset/translations/messages.pot
--config superset/translations/babel.cfg -k _ -k __ -k t -k tn -k tct
+```bash
+fabmanager babel-extract --target superset/translations --output superset/translations/messages.pot
--config superset/translations/babel.cfg -k _ -k __ -k t -k tn -k tct
+```
 
 You can then translate the strings gathered in files located under
 `superset/translation`, where there's one per language. For the translations
-to take effect, they need to be compiled using this command:
+to take effect:
 
-    fabmanager babel-compile --target superset/translations/
+```bash
+# In the case of JS translation, we need to convert the PO file into a JSON file, and we
need the global download of the npm package po2json.
+npm install -g po2json
+fabmanager babel-compile --target superset/translations
+# Convert the en PO file into a JSON file
+po2json -d superset -f jed1.x superset/translations/en/LC_MESSAGES/messages.po superset/translations/en/LC_MESSAGES/messages.json
+```
 
-In the case of JS translation, we need to convert the PO file into a JSON file, and we need
the global download of the npm package po2json.
-We need to be compiled using this command:
+If you get errors running `po2json`, you might be running the Ubuntu package with the same
+name, rather than the NodeJS package (they have a different format for the arguments). If
+there is a conflict, you may need to update your `PATH` environment variable or fully qualify
+the executable path (e.g. `/usr/local/bin/po2json` instead of `po2json`).
 
-    npm install po2json -g
+### Creating a new language dictionary
 
-Execute this command to convert the en PO file into a json file:
+To create a dictionary for a new language, run the following, where `LANGUAGE_CODE` is replaced
with
+the language code for your target language, e.g. `es` (see [Flask AppBuilder i18n documentation](https://flask-appbuilder.readthedocs.io/en/latest/i18n.html)
for more details):
 
-    po2json -d superset -f jed1.x superset/translations/en/LC_MESSAGES/messages.po superset/translations/en/LC_MESSAGES/messages.json
+```bash
+pip install -r superset/translations/requirements.txt
+pybabel init -i superset/translations/messages.pot -d superset/translations -l LANGUAGE_CODE
+```
 
-If you get errors running `po2json`, you might be running the ubuntu package with the same
-name rather than the nodejs package (they have a different format for the arguments). You
-need to be running the nodejs version, and so if there is a conflict you may need to point
-directly at `/usr/local/bin/po2json` rather than just `po2json`.
+Then, [extract strings for the new language](#extracting-new-strings-for-translation).
 
-## Adding new datasources
+## Tips
+
+### Adding a new datasource
 
 1. Create Models and Views for the datasource, add them under superset folder, like a new
my_models.py
     with models for cluster, datasources, columns and metrics and my_views.py with clustermodelview
     and datasourcemodelview.
 
-2. Create db migration files for the new models
+1. Create DB migration files for the new models
 
-3. Specify this variable to add the datasource model and from which module it is from in
config.py:
+1. Specify this variable to add the datasource model and from which module it is from in
config.py:
 
     For example:
 
-    `ADDITIONAL_MODULE_DS_MAP = {'superset.my_models': ['MyDatasource', 'MyOtherDatasource']}`
+    ```python
+    ADDITIONAL_MODULE_DS_MAP = {'superset.my_models': ['MyDatasource', 'MyOtherDatasource']}
+    ```
 
     This means it'll register MyDatasource and MyOtherDatasource in superset.my_models module
in the source registry.
 
-## Creating a new visualization type
+### Creating a new visualization type
 
 Here's an example as a Github PR with comments that describe what the
 different sections of the code do:
 https://github.com/apache/incubator-superset/pull/3013
 
-## Refresh documentation website
+### Adding a DB migration
 
-  Every once in a while we want to compile the documentation and publish it.
-  Here's how to do it.
+1. Alter the model you want to change. This example will add a `Column` Annotations model.
 
-```
-    # install doc dependencies
-    pip install -r docs/requirements.txt
+    [Example commit](https://github.com/apache/incubator-superset/commit/6c25f549384d7c2fc288451222e50493a7b14104)
 
-    # build the docs
-    python setup.py build_sphinx
+1. Generate the migration file
 
-    # copy html files to temp folder
-    cp -r docs/_build/html/ /tmp/tmp_superset_docs/
+    ```bash
+    superset db migrate -m 'add_metadata_column_to_annotation_model.py'
+    ```
 
-    # clone the docs repo
-    cd ~/
-    git clone https://git-wip-us.apache.org/repos/asf/incubator-superset-site.git
-
-    # copy
-    cp -r /tmp/tmp_superset_docs/ ~/incubator-superset-site.git/
-
-    # commit and push to `asf-site` branch
-    cd ~/incubator-superset-site.git/
-    git checkout asf-site
-    git add .
-    git commit -a -m "New doc version"
-    git push origin master
-```
+    This will generate a file in `migrations/version/{SHA}_this_will_be_in_the_migration_filename.py`.
 
-## Publishing a Pypi release
+    [Example commit](https://github.com/apache/incubator-superset/commit/d3e83b0fd572c9d6c1297543d415a332858e262)
 
-  We create a branch that goes along each minor release `0.24`
-  and micro releases get corresponding tags as in `0.24.0`. Git history should
-  never be altered in release branches.
-  Bug fixes and security-related patches get cherry-picked
-  (usually from master) as in `git cherry-pick -x {SHA}`.
+1. Upgrade the DB
 
-  Following a set of cherries being picked, a release can be pushed to
-  Pypi as follows:
+    ```bash
+    superset db upgrade
+    ```
 
-```
-    # branching off of master
-    git checkout -b 0.25
-
-    # cherry-picking a SHA
-    git cherry-pick -x f9d85bd2e1fd9bc233d19c76bed09467522b968a
-    # repeat with other SHAs, don't forget the -x
-
-    # source of thruth for release numbers live in package.json
-    vi superset/assets/package.json
-    # hard code release in file, commit to the release branch
-    git commit -a -m "0.25.0"
-
-    # create the release tag in the release branch
-    git tag 0.25.0
-    git push apache 0.25 --tags
-
-    # check travis to confirm the build succeeded as
-    # you shouldn't assume that a clean cherry will be clean
-    # when landing on a new sundae
-
-    # compile the JS, and push to pypi
-    # to run this part you'll need a pypi account and rights on the
-    # superset package. Committers that want to ship releases
-    # should have this access.
-    # You'll also need a `.pypirc` as specified here:
-    # http://peterdowns.com/posts/first-time-with-pypi.html
-    ./pypi_push.sh
-
-    # publish an update to the CHANGELOG.md for the right version range
-    # looking the latest CHANGELOG entry for the second argument
-    ./gen_changelog.sh 0.22.1 0.25.0
-    # this will overwrite the CHANGELOG.md with only the version range
-    # so you'll want to copy paste that on top of the previous CHANGELOG.md
-    # open a PR against `master`
-```
+    The output should look like this:
 
-  In the future we'll start publishing release candidates for minor releases
-  only, but typically not for micro release.
-  The process will be similar to the process described above, expect the
-  tags will be formatted `0.25.0rc1`, `0.25.0rc2`, ..., until consensus
-  is reached.
+    ```
+    INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
+    INFO  [alembic.runtime.migration] Will assume transactional DDL.
+    INFO  [alembic.runtime.migration] Running upgrade 1a1d627ebd8e -> 40a0a483dd12, add_metadata_column_to_annotation_model.py
+    ```
 
-  We should also have a Github PR label process to target the proper
-  release, and tooling helping keeping track of all the cherries and
-  target versions.
+1. Add column to view
 
-  For Apache releases, the process will be a bit heavier and should get
-  documented here. There will be extra steps for signing the binaries,
-  with a PGP key and providing MD5, Apache voting, as well as
-  publishing to Apache's SVN repository. View the ASF docs for more
-  information.
+    Since there is a new column, we need to add it to the AppBuilder Model view.
 
+    [Example commit](https://github.com/apache/incubator-superset/pull/5745/commits/6220966e2a0a0cf3e6d87925491f8920fe8a3458)
 
-## Merging DB migrations
+### Merging DB migrations
 
-When 2 db migrations collide, you'll get an error message like this one:
+When two DB migrations collide, you'll get an error message like this one:
 
 ```
-  alembic.util.exc.CommandError: Multiple head revisions are present for
-  given argument 'head'; please specify a specific target
-  revision, '<branchname>@head' to narrow to a specific head,
-  or 'heads' for all heads`
+alembic.util.exc.CommandError: Multiple head revisions are present for
+given argument 'head'; please specify a specific target
+revision, '<branchname>@head' to narrow to a specific head,
+or 'heads' for all heads`
 ```
 
-To fix it, first run `superset db heads`, this should list 2 or more
-migration hashes. Then run
-`superset db merge {PASTE_SHA1_HERE} {PASTE_SHA2_HERE}`. This will create
-a new merge migration. You can then `superset db upgrade` to this new
-checkpoint.
-
-
-## Running DB migration
-
-1. First alter the model you want to change. For example I want to add a `Column` Annotations
model.
-
-https://github.com/apache/incubator-superset/commit/6c25f549384d7c2fc288451222e50493a7b14104
+To fix it:
 
+1. Get the migration heads
 
-2. superset db migrate -m "this_will_be_in_the_migration_filename"
+    ```bash
+    superset db heads
+    ```
 
-For our example we'll be running this command:
-```
-superset db migrate -m "add_metadata_column_to_annotation_model.py"
-```
+    This should list two or more migration hashes.
 
-This will generate a file in `superset/migrations/version/{SHA}_this_will_be_in_the_migration_filename.py`
+1. Create a new merge migration
 
-https://github.com/apache/incubator-superset/commit/d3e83b0fd572c9d6c1297543d415a332858e262
+    ```bash
+    superset db merge {HASH1} {HASH2}
+    ```
 
-3. Run `superset db upgrade`
+1. Upgrade the DB to the new checkpoint
 
-The output should look like this:
-```
-INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
-INFO  [alembic.runtime.migration] Will assume transactional DDL.
-INFO  [alembic.runtime.migration] Running upgrade 1a1d627ebd8e -> 40a0a483dd12, add_metadata_column_to_annotation_model.py
-```
-
-4. Add column to view
-Since there is a new column, we need to add it to the AppBuilder Model view.
-
-https://github.com/apache/incubator-superset/pull/5745/commits/6220966e2a0a0cf3e6d87925491f8920fe8a3458
+    ```bash
+    superset db upgrade
+    ```
diff --git a/RELEASING.md b/RELEASING.md
new file mode 100644
index 0000000..dbf780e
--- /dev/null
+++ b/RELEASING.md
@@ -0,0 +1,93 @@
+## Refresh documentation website
+
+Every once in a while we want to compile the documentation and publish it.
+Here's how to do it.
+
+```bash
+# install doc dependencies
+pip install -r docs/requirements.txt
+
+# build the docs
+python setup.py build_sphinx
+
+# copy html files to temp folder
+cp -r docs/_build/html/ /tmp/tmp_superset_docs/
+
+# clone the docs repo
+cd ~/
+git clone https://git-wip-us.apache.org/repos/asf/incubator-superset-site.git
+
+# copy
+cp -r /tmp/tmp_superset_docs/ ~/incubator-superset-site.git/
+
+# commit and push to `asf-site` branch
+cd ~/incubator-superset-site.git/
+git checkout asf-site
+git add .
+git commit -a -m "New doc version"
+git push origin master
+```
+
+## Publishing a PyPI release
+
+We create a branch that goes along each minor release `0.24`
+and micro releases get corresponding tags as in `0.24.0`. Git history should
+never be altered in release branches.
+Bug fixes and security-related patches get cherry-picked
+(usually from master) as in `git cherry-pick -x {SHA}`.
+
+Following a set of cherries being picked, a release can be pushed to
+PyPI as follows:
+
+```bash
+# branching off of master
+git checkout -b 0.25
+
+# cherry-picking a SHA
+git cherry-pick -x f9d85bd2e1fd9bc233d19c76bed09467522b968a
+# repeat with other SHAs, don't forget the -x
+
+# source of thruth for release numbers live in package.json
+vi superset/assets/package.json
+# hard code release in file, commit to the release branch
+git commit -a -m "0.25.0"
+
+# create the release tag in the release branch
+git tag 0.25.0
+git push apache 0.25 --tags
+
+# check travis to confirm the build succeeded as
+# you shouldn't assume that a clean cherry will be clean
+# when landing on a new sundae
+
+# compile the JS, and push to pypi
+# to run this part you'll need a pypi account and rights on the
+# superset package. Committers that want to ship releases
+# should have this access.
+# You'll also need a `.pypirc` as specified here:
+# http://peterdowns.com/posts/first-time-with-pypi.html
+./pypi_push.sh
+
+# publish an update to the CHANGELOG.md for the right version range
+# looking the latest CHANGELOG entry for the second argument
+./gen_changelog.sh 0.22.1 0.25.0
+# this will overwrite the CHANGELOG.md with only the version range
+# so you'll want to copy paste that on top of the previous CHANGELOG.md
+# open a PR against `master`
+```
+
+In the future we'll start publishing release candidates for minor releases
+only, but typically not for micro release.
+The process will be similar to the process described above, expect the
+tags will be formatted `0.25.0rc1`, `0.25.0rc2`, ..., until consensus
+is reached.
+
+We should also have a Github PR label process to target the proper
+release, and tooling helping keeping track of all the cherries and
+target versions.
+
+For Apache releases, the process will be a bit heavier and should get
+documented here. There will be extra steps for signing the binaries,
+with a PGP key and providing MD5, Apache voting, as well as
+publishing to Apache's SVN repository. View the ASF docs for more
+information.
diff --git a/docs/installation.rst b/docs/installation.rst
index 963d273..69fc0a4 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -672,20 +672,12 @@ and run via: ::
 
     celery flower --app=superset.sql_lab:celery_app
 
-Making your own build
+Building from source
 ---------------------
 
-For more advanced users, you may want to build Superset from sources. That
+More advanced users may want to build Superset from sources. That
 would be the case if you fork the project to add features specific to
-your environment.::
-
-    # assuming $SUPERSET_HOME as the root of the repo
-    cd $SUPERSET_HOME/superset/assets
-    yarn
-    yarn run build
-    cd $SUPERSET_HOME
-    python setup.py install
-
+your environment. See `CONTRIBUTING.md <https://github.com/apache/incubator-superset/blob/master/CONTRIBUTING.md#local-development>`_.
 
 Blueprints
 ----------
diff --git a/superset/translations/requirements.txt b/superset/translations/requirements.txt
new file mode 100644
index 0000000..a53c871
--- /dev/null
+++ b/superset/translations/requirements.txt
@@ -0,0 +1 @@
+Babel==2.5.3


Mime
View raw message