superset-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maximebeauche...@apache.org
Subject [13/14] incubator-superset-site git commit: Initial commit - Superset docs as of 0.18.5
Date Tue, 27 Jun 2017 20:12:39 GMT
http://git-wip-us.apache.org/repos/asf/incubator-superset-site/blob/886f04b0/_sources/installation.txt
----------------------------------------------------------------------
diff --git a/_sources/installation.txt b/_sources/installation.txt
new file mode 100644
index 0000000..10aa7d7
--- /dev/null
+++ b/_sources/installation.txt
@@ -0,0 +1,475 @@
+Installation & Configuration
+============================
+
+Getting Started
+---------------
+
+Superset is tested against Python ``2.7`` and Python ``3.4``.
+Airbnb currently uses 2.7.* in production. We do not plan on supporting
+Python ``2.6``.
+
+
+OS dependencies
+---------------
+
+Superset stores database connection information in its metadata database.
+For that purpose, we use the ``cryptography`` Python library to encrypt
+connection passwords. Unfortunately this library has OS level dependencies.
+
+You may want to attempt the next step
+("Superset installation and initialization") and come back to this step if
+you encounter an error.
+
+Here's how to install them:
+
+For **Debian** and **Ubuntu**, the following command will ensure that
+the required dependencies are installed: ::
+
+    sudo apt-get install build-essential libssl-dev libffi-dev python-dev python-pip libsasl2-dev
libldap2-dev
+
+For **Fedora** and **RHEL-derivatives**, the following command will ensure
+that the required dependencies are installed: ::
+
+    sudo yum upgrade python-setuptools
+    sudo yum install gcc gcc-c++ libffi-devel python-devel python-pip python-wheel openssl-devel
libsasl2-devel openldap-devel
+
+**OSX**, system python is not recommended. brew's python also ships with pip  ::
+
+    brew install pkg-config libffi openssl python
+    env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include"
pip install cryptography==1.7.2
+
+**Windows** isn't officially supported at this point, but if you want to
+attempt it, download `get-pip.py <https://bootstrap.pypa.io/get-pip.py>`_, and run
``python get-pip.py`` which may need admin access. Then run the following: ::
+
+    C:\> pip install cryptography
+
+    # You may also have to create C:\Temp
+    C:\> md C:\Temp
+
+Python virtualenv
+-----------------
+It is recommended to install Superset inside a virtualenv. Python 3 already ships virtualenv,
for
+Python 2 you need to install it. If it's packaged for your operating systems install it from
there
+otherwise you can install from pip: ::
+
+    pip install virtualenv
+
+You can create and activate a virtualenv by: ::
+
+    # virtualenv is shipped in Python 3 as pyvenv
+    virtualenv venv
+    . ./venv/bin/activate
+
+On windows the syntax for activating it is a bit different: ::
+
+    venv\Scripts\activate
+
+Once you activated your virtualenv everything you are doing is confined inside the virtualenv.
+To exit a virtualenv just type ``deactivate``.
+
+Python's setup tools and pip
+----------------------------
+Put all the chances on your side by getting the very latest ``pip``
+and ``setuptools`` libraries.::
+
+    pip install --upgrade setuptools pip
+
+Superset installation and initialization
+----------------------------------------
+Follow these few simple steps to install Superset.::
+
+    # Install superset
+    pip install superset
+
+    # Create an admin user (you will be prompted to set username, first and last name before
setting a password)
+    fabmanager create-admin --app superset
+
+    # Initialize the database
+    superset db upgrade
+
+    # Load some data to play with
+    superset load_examples
+
+    # Create default roles and permissions
+    superset init
+
+    # Start the web server on port 8088, use -p to bind to another port
+    superset runserver
+
+    # To start a development web server, use the -d switch
+    # superset runserver -d
+
+
+After installation, you should be able to point your browser to the right
+hostname:port `http://localhost:8088 <http://localhost:8088>`_, login using
+the credential you entered while creating the admin account, and navigate to
+`Menu -> Admin -> Refresh Metadata`. This action should bring in all of
+your datasources for Superset to be aware of, and they should show up in
+`Menu -> Datasources`, from where you can start playing with your data!
+
+Please note that *gunicorn*, Superset default application server, does not
+work on Windows so you need to use the development web server.
+The development web server though is not intended to be used on production systems
+so better use a supported platform that can run *gunicorn*.
+
+Configuration behind a load balancer
+------------------------------------
+
+If you are running superset behind a load balancer or reverse proxy (e.g. NGINX
+or ELB on AWS), you may need to utilise a healthcheck endpoint so that your
+load balancer knows if your superset instance is running. This is provided
+at ``/health`` which will return a 200 response containing "OK" if the
+webserver is running.
+
+If the load balancer is inserting X-Forwarded-For/X-Forwarded-Proto headers, you
+should set `ENABLE_PROXY_FIX = True` in the superset config file to extract and use
+the headers.
+
+
+Configuration
+-------------
+
+To configure your application, you need to create a file (module)
+``superset_config.py`` and make sure it is in your PYTHONPATH. Here are some
+of the parameters you can copy / paste in that configuration module: ::
+
+    #---------------------------------------------------------
+    # Superset specific config
+    #---------------------------------------------------------
+    ROW_LIMIT = 5000
+    SUPERSET_WORKERS = 4
+
+    SUPERSET_WEBSERVER_PORT = 8088
+    #---------------------------------------------------------
+
+    #---------------------------------------------------------
+    # Flask App Builder configuration
+    #---------------------------------------------------------
+    # Your App secret key
+    SECRET_KEY = '\2\1thisismyscretkey\1\2\e\y\y\h'
+
+    # The SQLAlchemy connection string to your database backend
+    # This connection defines the path to the database that stores your
+    # superset metadata (slices, connections, tables, dashboards, ...).
+    # Note that the connection information to connect to the datasources
+    # you want to explore are managed directly in the web UI
+    SQLALCHEMY_DATABASE_URI = 'sqlite:////path/to/superset.db'
+
+    # Flask-WTF flag for CSRF
+    WTF_CSRF_ENABLED = True
+
+    # Set this API key to enable Mapbox visualizations
+    MAPBOX_API_KEY = ''
+
+This file also allows you to define configuration parameters used by
+Flask App Builder, the web framework used by Superset. Please consult
+the `Flask App Builder Documentation
+<http://flask-appbuilder.readthedocs.org/en/latest/config.html>`_
+for more information on how to configure Superset.
+
+Please make sure to change:
+
+* *SQLALCHEMY_DATABASE_URI*, by default it is stored at *~/.superset/superset.db*
+* *SECRET_KEY*, to a long random string
+
+Database dependencies
+---------------------
+
+Superset does not ship bundled with connectivity to databases, except
+for Sqlite, which is part of the Python standard library.
+You'll need to install the required packages for the database you
+want to use as your metadata database as well as the packages needed to
+connect to the databases you want to access through Superset.
+
+Here's a list of some of the recommended packages.
+
++---------------+-------------------------------------+-------------------------------------------------+
+| database      | pypi package                        | SQLAlchemy URI prefix           
               |
++===============+=====================================+=================================================+
+|  MySQL        | ``pip install mysqlclient``         | ``mysql://``                    
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  Postgres     | ``pip install psycopg2``            | ``postgresql+psycopg2://``      
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  Presto       | ``pip install pyhive``              | ``presto://``                   
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  Oracle       | ``pip install cx_Oracle``           | ``oracle://``                   
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  sqlite       |                                     | ``sqlite://``                   
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  Redshift     | ``pip install sqlalchemy-redshift`` | ``postgresql+psycopg2://``      
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  MSSQL        | ``pip install pymssql``             | ``mssql://``                    
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  Impala       | ``pip install impyla``              | ``impala://``                   
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  SparkSQL     | ``pip install pyhive``              | ``jdbc+hive://``                
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  Greenplum    | ``pip install psycopg2``            | ``postgresql+psycopg2://``      
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  Athena       | ``pip install "PyAthenaJDBC>1.0.9"``| ``awsathena+jdbc://``        
                  |
++---------------+-------------------------------------+-------------------------------------------------+
+|  Vertica      | ``pip install                       |  ``vertica+vertica_python://``  
               |
+|               | sqlalchemy-vertica-python``         |                                 
               |
++---------------+-------------------------------------+-------------------------------------------------+
+|  ClickHouse   | ``pip install                       | ``clickhouse://``               
               |
+|               | sqlalchemy-clickhouse``             |                                 
               |
++---------------+-------------------------------------+-------------------------------------------------+
+
+Note that many other database are supported, the main criteria being the
+existence of a functional SqlAlchemy dialect and Python driver. Googling
+the keyword ``sqlalchemy`` in addition of a keyword that describes the
+database you want to connect to should get you to the right place.
+
+(AWS) Athena
+------------
+
+This currently relies on an unreleased future version of `PyAthenaJDBC <https://github.com/laughingman7743/PyAthenaJDBC>`_.
If you're adventurous or simply impatient, you can install directly from git: ::
+
+    pip install git+https://github.com/laughingman7743/PyAthenaJDBC@support_sqlalchemy
+
+The connection string for Athena looks like this ::
+
+    awsathena+jdbc://{aws_access_key_id}:{aws_secret_access_key}@athena.{region_name}.amazonaws.com/{schema_name}?s3_staging_dir={s3_staging_dir}&...
+
+Where you need to escape/encode at least the s3_staging_dir, i.e., ::
+
+    s3://... -> s3%3A//...
+
+
+Caching
+-------
+
+Superset uses `Flask-Cache <https://pythonhosted.org/Flask-Cache/>`_ for
+caching purpose. Configuring your caching backend is as easy as providing
+a ``CACHE_CONFIG``, constant in your ``superset_config.py`` that
+complies with the Flask-Cache specifications.
+
+Flask-Cache supports multiple caching backends (Redis, Memcached,
+SimpleCache (in-memory), or the local filesystem). If you are going to use
+Memcached please use the `pylibmc` client library as `python-memcached` does
+not handle storing binary data correctly. If you use Redis, please install
+the `redis <https://pypi.python.org/pypi/redis>`_ Python package: ::
+
+    pip install redis
+
+For setting your timeouts, this is done in the Superset metadata and goes
+up the "timeout searchpath", from your slice configuration, to your
+data source's configuration, to your database's and ultimately falls back
+into your global default defined in ``CACHE_CONFIG``.
+
+
+Deeper SQLAlchemy integration
+-----------------------------
+
+It is possible to tweak the database connection information using the
+parameters exposed by SQLAlchemy. In the ``Database`` edit view, you will
+find an ``extra`` field as a ``JSON`` blob.
+
+.. image:: _static/img/tutorial/add_db.png
+   :scale: 30 %
+
+This JSON string contains extra configuration elements. The ``engine_params``
+object gets unpacked into the
+`sqlalchemy.create_engine <http://docs.sqlalchemy.org/en/latest/core/engines.html#sqlalchemy.create_engine>`_
call,
+while the ``metadata_params`` get unpacked into the
+`sqlalchemy.MetaData <http://docs.sqlalchemy.org/en/rel_1_0/core/metadata.html#sqlalchemy.schema.MetaData>`_
call. Refer to the SQLAlchemy docs for more information.
+
+
+Schemas (Postgres & Redshift)
+-----------------------------
+
+Postgres and Redshift, as well as other database,
+use the concept of **schema** as a logical entity
+on top of the **database**. For Superset to connect to a specific schema,
+there's a **schema** parameter you can set in the table form.
+
+
+SSL Access to databases
+-----------------------
+This example worked with a MySQL database that requires SSL. The configuration
+may differ with other backends. This is what was put in the ``extra``
+parameter ::
+
+    {
+        "metadata_params": {},
+        "engine_params": {
+              "connect_args":{
+                  "sslmode":"require",
+                  "sslrootcert": "/path/to/my/pem"
+            }
+         }
+    }
+
+
+Druid
+-----
+
+* From the UI, enter the information about your clusters in the
+  ``Admin->Clusters`` menu by hitting the + sign.
+
+* Once the Druid cluster connection information is entered, hit the
+  ``Admin->Refresh Metadata`` menu item to populate
+
+* Navigate to your datasources
+
+Note that you can run the ``superset refresh_druid`` command to refresh the
+metadata from your Druid cluster(s)
+
+
+CORS
+-----
+
+The extra CORS Dependency must be installed:
+
+    superset[cors]
+
+
+The following keys in `superset_config.py` can be specified to configure CORS:
+
+
+* ``ENABLE_CORS``: Must be set to True in order to enable CORS
+* ``CORS_OPTIONS``: options passed to Flask-CORS (`documentation <http://flask-cors.corydolphin.com/en/latest/api.html#extension>`)
+
+
+MIDDLEWARE
+----------
+
+Superset allows you to add your own middleware. To add your own middleware, update the ``ADDITIONAL_MIDDLEWARE``
key in
+your `superset_config.py`. ``ADDITIONAL_MIDDLEWARE`` should be a list of your additional
middleware classes.
+
+For example, to use AUTH_REMOTE_USER from behind a proxy server like nginx, you have to add
a simple middleware class to
+add the value of ``HTTP_X_PROXY_REMOTE_USER`` (or any other custom header from the proxy)
to Gunicorn's ``REMOTE_USER``
+environment variable: ::
+
+    class RemoteUserMiddleware(object):
+        def __init__(self, app):
+            self.app = app
+        def __call__(self, environ, start_response):
+            user = environ.pop('HTTP_X_PROXY_REMOTE_USER', None)
+            environ['REMOTE_USER'] = user
+            return self.app(environ, start_response)
+
+    ADDITIONAL_MIDDLEWARE = [RemoteUserMiddleware, ]
+
+*Adapted from http://flask.pocoo.org/snippets/69/*
+
+
+Upgrading
+---------
+
+Upgrading should be as straightforward as running::
+
+    pip install superset --upgrade
+    superset db upgrade
+    superset init
+
+SQL Lab
+-------
+SQL Lab is a powerful SQL IDE that works with all SQLAlchemy compatible
+databases. By default, queries are executed in the scope of a web
+request so they
+may eventually timeout as queries exceed the maximum duration of a web
+request in your environment, whether it'd be a reverse proxy or the Superset
+server itself.
+
+On large analytic databases, it's common to run queries that
+execute for minutes or hours.
+To enable support for long running queries that
+execute beyond the typical web request's timeout (30-60 seconds), it is
+necessary to configure an asynchronous backend for Superset which consist of:
+
+* one or many Superset worker (which is implemented as a Celery worker), and
+  can be started with the ``superset worker`` command, run
+  ``superset worker --help`` to view the related options
+* a celery broker (message queue) for which we recommend using Redis
+  or RabbitMQ
+* a results backend that defines where the worker will persist the query
+  results
+
+Configuring Celery requires defining a ``CELERY_CONFIG`` in your
+``superset_config.py``. Both the worker and web server processes should
+have the same configuration.
+
+.. code-block:: python
+
+ 	class CeleryConfig(object):
+    	BROKER_URL = 'redis://localhost:6379/0'
+		CELERY_IMPORTS = ('superset.sql_lab', )
+		CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
+		CELERY_ANNOTATIONS = {'tasks.add': {'rate_limit': '10/s'}}
+
+  	CELERY_CONFIG = CeleryConfig
+
+To setup a result backend, you need to pass an instance of a derivative
+of ``werkzeug.contrib.cache.BaseCache`` to the ``RESULTS_BACKEND``
+configuration key in your ``superset_config.py``. It's possible to use
+Memcached, Redis, S3 (https://pypi.python.org/pypi/s3werkzeugcache),
+memory or the file system (in a single server-type setup or for testing),
+or to write your own caching interface. Your ``superset_config.py`` may
+look something like:
+
+.. code-block:: python
+
+	# On S3
+  	from s3cache.s3cache import S3Cache
+  	S3_CACHE_BUCKET = 'foobar-superset'
+	S3_CACHE_KEY_PREFIX = 'sql_lab_result'
+  	RESULTS_BACKEND = S3Cache(S3_CACHE_BUCKET, S3_CACHE_KEY_PREFIX)
+
+	# On Redis
+    from werkzeug.contrib.cache import RedisCache
+    RESULTS_BACKEND = RedisCache(
+        host='localhost', port=6379, key_prefix='superset_results')
+
+
+Also note that SQL Lab supports Jinja templating in queries, and that it's
+possible to overload
+the default Jinja context in your environment by defining the
+``JINJA_CONTEXT_ADDONS`` in your superset configuration. Objects referenced
+in this dictionary are made available for users to use in their SQL.
+
+.. code-block:: python
+
+    JINJA_CONTEXT_ADDONS = {
+        'my_crazy_macro': lambda x: x*2,
+    }
+
+
+Making your own build
+---------------------
+
+For more advanced users, you may want to build Superset from sources. That
+would be the case if you fork the project to add features specific to
+your environment.::
+
+    # assuming $SUPERSET_HOME as the root of the repo
+    cd $SUPERSET_HOME/superset/assets
+    npm install
+    npm run build
+    cd $SUPERSET_HOME
+    python setup.py install
+
+
+Blueprints
+----------
+
+`Blueprints are Flask's reusable apps <http://flask.pocoo.org/docs/0.12/blueprints/>`_.
+Superset allows you to specify an array of Blueprints
+in your ``superset_config`` module. Here's
+an example on how this can work with a simple Blueprint. By doing
+so, you can expect Superset to serve a page that says "OK"
+at the ``/simple_page`` url. This can allow you to run other things such
+as custom data visualization applications alongside Superset, on the
+same server.
+
+..code ::
+
+    from flask import Blueprint
+    simple_page = Blueprint('simple_page', __name__,
+                                    template_folder='templates')
+    @simple_page.route('/', defaults={'page': 'index'})
+    @simple_page.route('/<page>')
+    def show(page):
+        return "Ok"
+
+    BLUEPRINTS = [simple_page]

http://git-wip-us.apache.org/repos/asf/incubator-superset-site/blob/886f04b0/_sources/security.txt
----------------------------------------------------------------------
diff --git a/_sources/security.txt b/_sources/security.txt
new file mode 100644
index 0000000..b5d5b63
--- /dev/null
+++ b/_sources/security.txt
@@ -0,0 +1,161 @@
+Security
+========
+Security in Superset is handled by Flask AppBuilder (FAB). FAB is a
+"Simple and rapid application development framework, built on top of Flask.".
+FAB provides authentication, user management, permissions and roles.
+
+
+Provided Roles
+--------------
+Superset ships with a set of roles that are handled by Superset itself.
+You can assume that these roles will stay up-to-date as Superset evolves.
+Even though it's possible for ``Admin`` usrs to do so, it is not recommended
+that you alter these roles in any way by removing
+or adding permissions to them as these roles will be re-synchronized to
+their original values as you run your next ``superset init`` command.
+
+Since it's not recommended to alter the roles described here, it's right
+to assume that your security strategy should be to compose user access based
+on these base roles and roles that you create. For instance you could
+create a role ``Financial Analyst`` that would be made of set of permissions
+to a set of data sources (tables) and/or databases. Users would then be
+granted ``Gamma``, ``Financial Analyst``, and perhaps ``sql_lab``.
+
+Admin
+"""""
+Admins have all possible rights, including granting or revoking rights from
+other users and altering other people's slices and dashboards.
+
+Alpha
+"""""
+Alpha have access to all data sources, but they cannot grant or revoke access
+from other users. They are also limited to altering the objects that they
+own. Alpha users can add and alter data sources.
+
+Gamma
+"""""
+Gamma have limited access. They can only consume data coming from data sources
+they have been giving access to through another complementary role.
+They only have access to view the slices and
+dashboards made from data sources that they have access to. Currently Gamma
+users are not able to alter or add data sources. We assume that they are
+mostly content consumers, though they can create slices and dashboards.
+
+Also note that when Gamma users look at the dashboards and slices list view,
+they will only see the objects that they have access to.
+
+sql_lab
+"""""""
+The ``sql_lab`` role grants access to SQL Lab. Note that while ``Admin``
+users have access to all databases by default, both ``Alpha`` and ``Gamma``
+users need to be given access on a per database basis.
+
+Public
+""""""
+It's possible to allow logged out users to access some Superset features.
+
+By setting ``PUBLIC_ROLE_LIKE_GAMMA = True`` in your ``superset_config.py``,
+you grant public role the same set of permissions as for the GAMMA role.
+This is useful if one wants to enable anonymous users to view
+dashboards. Explicit grant on specific datasets is still required, meaning
+that you need to edit the ``Public`` role and add the Public data sources
+to the role manually.
+
+
+Managing Gamma per data source access
+-------------------------------------
+Here's how to provide users access to only specific datasets. First make
+sure the users with limited access have [only] the Gamma role assigned to
+them. Second, create a new role (``Menu -> Security -> List Roles``) and
+click the ``+`` sign.
+
+.. image:: _static/img/create_role.png
+   :scale: 50 %
+
+This new window allows you to give this new role a name, attribute it to users
+and select the tables in the ``Permissions`` dropdown. To select the data
+sources you want to associate with this role, simply click in the dropdown
+and use the typeahead to search for your table names.
+
+You can then confirm with your Gamma users that they see the objects
+(dashboards and slices) associated with the tables related to their roles.
+
+
+Customizing
+-----------
+
+The permissions exposed by FAB are very granular and allow for a great level
+of customization. FAB creates many permissions automagically for each model
+that is create (can_add, can_delete, can_show, can_edit, ...) as well as for
+each view. On top of that, Superset can expose more granular permissions like
+``all_datasource_access``.
+
+We do not recommend altering the 3 base roles as there
+are a set of assumptions that Superset build upon. It is possible though for
+you to create your own roles, and union them to existing ones.
+
+Permissions
+"""""""""""
+
+Roles are composed of a set of permissions, and Superset has many categories
+of permissions. Here are the different categories of permissions:
+
+- **Model & action**: models are entities like ``Dashboard``,
+  ``Slice``, or ``User``. Each model has a fixed set of permissions, like
+  ``can_edit``, ``can_show``, ``can_delete``, ``can_list``, ``can_add``, and
+  so on. By adding ``can_delete on Dashboard`` to a role, and granting that
+  role to a user, this user will be able to delete dashboards.
+- **Views**: views are individual web pages, like the ``explore`` view or the
+  ``SQL Lab`` view. When granted to a user, he/she will see that view in
+  the its menu items, and be able to load that page.
+- **Data source**: For each data source, a permission is created. If the user
+  does not have the ``all_datasource_access`` permission granted, the user
+  will only be able to see Slices or explore the data sources that are granted
+  to them
+- **Database**: Granting access to a database allows for the user to access
+  all data sources within that database, and will enable the user to query
+  that database in SQL Lab, provided that the SQL Lab specific permission
+  have been granted to the user
+
+
+Restricting access to a subset of data sources
+""""""""""""""""""""""""""""""""""""""""""""""
+
+The best way to go is probably to give user ``Gamma`` plus one or many other
+roles that would add access to specific data sources. We recommend that you
+create individual roles for each access profile. Say people in your finance
+department might have access to a set of databases and data sources, and
+these permissions can be consolidated in a single role. Users with this
+profile then need to be attributed ``Gamma`` as a foundation to the models
+and views they can access, and that ``Finance`` role that is a collection
+of permissions to data objects.
+
+One user can have many roles, so a finance executive could be granted
+``Gamma``, ``Finance``, and perhaps another ``Executive`` role that gather
+a set of data sources that power dashboards only made available to executives.
+When looking at its dashboard list, this user will only see the
+list of dashboards it has access to, based on the roles and
+permissions that were attributed.
+
+
+Restricting the access to some metrics
+""""""""""""""""""""""""""""""""""""""
+
+Sometimes some metrics are relatively sensitive (e.g. revenue).
+We may want to restrict those metrics to only a few roles.
+For example, assumed there is a metric ``[cluster1].[datasource1].[revenue]``
+and only Admin users are allowed to see it. Here’s how to restrict the access.
+
+1. Edit the datasource (``Menu -> Source -> Druid datasources -> edit the
+   record "datasource1"``) and go to the tab ``List Druid Metric``. Check
+   the checkbox ``Is Restricted`` in the row of the metric ``revenue``.
+
+2. Edit the role (``Menu -> Security -> List Roles -> edit the record
+   “Admin”``), in the permissions field, type-and-search the permission
+   ``metric access on [cluster1].[datasource1].[revenue] (id: 1)``, then
+   click the Save button on the bottom of the page.
+
+Any users without the permission will see the error message
+*Access to the metrics denied: revenue (Status: 500)* in the slices.
+It also happens when the user wants to access a post-aggregation metric that
+is dependent on revenue.

http://git-wip-us.apache.org/repos/asf/incubator-superset-site/blob/886f04b0/_sources/sqllab.txt
----------------------------------------------------------------------
diff --git a/_sources/sqllab.txt b/_sources/sqllab.txt
new file mode 100644
index 0000000..a1da6c7
--- /dev/null
+++ b/_sources/sqllab.txt
@@ -0,0 +1,64 @@
+SQL Lab
+=======
+
+SQL Lab is a modern, feature-rich SQL IDE written in
+`React <https://facebook.github.io/react/>`_.
+
+
+Feature Overview
+----------------
+- Connects to just about any database backend
+- A multi-tab environment to work on multiple queries at a time
+- A smooth flow to visualize your query results using Superset's rich
+  visualization capabilities
+- Browse database metadata: tables, columns, indexes, partitions
+- Support for long-running queries
+
+  - uses the `Celery distributed queue <http://www.python.org/>`_
+    to dispatch query handling to workers
+  - supports defining a "results backend" to persist query results
+
+- A search engine to find queries executed in the past
+- Supports templating using the
+  `Jinja templating language <http://jinja.pocoo.org/docs/dev/>`_
+  which allows for using macros in your SQL code
+
+Extra features
+--------------
+- Hit ``alt + enter`` as a keyboard shortcut to run your query
+
+Templating with Jinja
+---------------------
+
+.. code-block:: sql
+
+    SELECT *
+    FROM some_table
+    WHERE partition_key = '{{ presto.latest_partition('some_table') }}'
+
+Templating unleashes the power and capabilities of a
+programming language within your SQL code.
+
+Templates can also be used to write generic queries that are
+parameterized so they can be re-used easily.
+
+
+Available macros
+''''''''''''''''
+
+We expose certain modules from Python's standard library in
+Superset's Jinja context:
+- ``time``: ``time``
+- ``datetime``: ``datetime.datetime``
+- ``uuid``: ``uuid``
+- ``random``: ``random``
+- ``relativedelta``: ``dateutil.relativedelta.relativedelta``
+- more to come!
+
+`Jinja's builtin filters <http://jinja.pocoo.org/docs/dev/templates/>`_ can be also
be applied where needed.
+
+
+.. autoclass:: superset.jinja_context.PrestoTemplateProcessor
+    :members:
+
+.. autofunction:: superset.jinja_context.url_param

http://git-wip-us.apache.org/repos/asf/incubator-superset-site/blob/886f04b0/_sources/tutorial.txt
----------------------------------------------------------------------
diff --git a/_sources/tutorial.txt b/_sources/tutorial.txt
new file mode 100644
index 0000000..695057c
--- /dev/null
+++ b/_sources/tutorial.txt
@@ -0,0 +1,308 @@
+Tutorial for Superset Administrators
+====================================
+
+This tutorial targets a Superset administrator: someone configuring Superset 
+for an organization on behalf of users. We'll show you how to connect Superset 
+to a new database and configure a table in that database for analysis. You'll 
+also explore the data you've exposed and add a visualization to a dashboard 
+so that you get a feel for the end-to-end user experience.
+
+Connecting to a new database
+----------------------------
+
+We assume you already have a database configured and can connect to it from the 
+instance on which you’re running Superset. If you’re just testing Superset and 
+want to explore sample data, you can load some 
+`sample PostgreSQL datasets <https://wiki.postgresql.org/wiki/Sample_Databases>`_
+into a fresh DB, or configure the 
+`example weather data <https://github.com/dylburger/noaa-ghcn-weather-data>`_
+we use here.
+
+Under the **Sources** menu, select the *Databases* option:
+
+.. image:: _static/img/tutorial/tutorial_01_sources_database.png
+   :scale: 70%
+
+On the resulting page, click on the green plus sign, near the top left:
+
+.. image:: _static/img/tutorial/tutorial_02_add_database.png
+   :scale: 70%
+
+You can configure a number of advanced options on this page, but for 
+this walkthrough, you’ll only need to do **two things**:
+
+1. Name your database connection:
+
+.. image:: _static/img/tutorial/tutorial_03_database_name.png
+   :scale: 70%
+
+2. Provide the SQLAlchemy Connection URI and test the connection:
+
+.. image:: _static/img/tutorial/tutorial_04_sqlalchemy_connection_string.png
+   :scale: 70%
+
+This example shows the connection string for our test weather database. 
+As noted in the text below the URI, you should refer to the SQLAlchemy 
+documentation on 
+`creating new connection URIs <http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html#database-urls>`_
+for your target database.
+
+Click the **Test Connection** button to confirm things work end to end. 
+Once Superset can successfully connect and authenticate, you should see 
+a popup like this:
+
+.. image:: _static/img/tutorial/tutorial_05_connection_popup.png
+   :scale: 50%
+
+Moreover, you should also see the list of tables Superset can read from 
+the schema you’re connected to, at the bottom of the page:
+
+.. image:: _static/img/tutorial/tutorial_06_list_of_tables.png
+   :scale: 70%
+
+If the connection looks good, save the configuration by clicking the **Save** 
+button at the bottom of the page:
+
+.. image:: _static/img/tutorial/tutorial_07_save_button.png
+   :scale: 70%
+
+Adding a new table
+------------------
+
+Now that you’ve configured a database, you’ll need to add specific tables 
+to Superset that you’d like to query.
+
+Under the **Sources** menu, select the *Tables* option:
+
+.. image:: _static/img/tutorial/tutorial_08_sources_tables.png
+   :scale: 70%
+
+On the resulting page, click on the green plus sign, near the top left:
+
+.. image:: _static/img/tutorial/tutorial_09_add_new_table.png
+   :scale: 70%
+
+You only need a few pieces of information to add a new table to Superset:
+
+* The name of the table
+
+.. image:: _static/img/tutorial/tutorial_10_table_name.png
+   :scale: 70%
+
+* The target database from the **Database** drop-down menu (i.e. the one 
+  you just added above)
+
+.. image:: _static/img/tutorial/tutorial_11_choose_db.png
+   :scale: 70%
+
+* Optionally, the database schema. If the table exists in the “default” schema 
+  (e.g. the *public* schema in PostgreSQL or Redshift), you can leave the schema 
+  field blank.
+
+Click on the **Save** button to save the configuration:
+
+.. image:: _static/img/tutorial/tutorial_07_save_button.png
+   :scale: 70%
+
+When redirected back to the list of tables, you should see a message indicating 
+that your table was created:
+
+.. image:: _static/img/tutorial/tutorial_12_table_creation_success_msg.png
+   :scale: 70%
+
+This message also directs you to edit the table configuration. We’ll edit a limited 
+portion of the configuration now - just to get you started - and leave the rest for 
+a more advanced tutorial.
+
+Click on the edit button next to the table you’ve created:
+
+.. image:: _static/img/tutorial/tutorial_13_edit_table_config.png
+   :scale: 70%
+
+On the resulting page, click on the **List Table Column** tab. Here, you’ll define the

+way you can use specific columns of your table when exploring your data. We’ll run 
+through these options to describe their purpose:
+
+* If you want users to group metrics by a specific field, mark it as **Groupable**.
+* If you need to filter on a specific field, mark it as **Filterable**.
+* Is this field something you’d like to get the distinct count of? Check the **Count 
+  Distinct** box.
+* Is this a metric you want to sum, or get basic summary statistics for? The **Sum**, 
+  **Min**, and **Max** columns will help.
+* The **is temporal** field should be checked for any date or time fields. We’ll cover

+  how this manifests itself in analyses in a moment.
+
+Here’s how we’ve configured fields for the weather data. Even for measures like the 
+weather measurements (precipitation, snowfall, etc.), it’s ideal to group and filter 
+by these values:
+
+.. image:: _static/img/tutorial/tutorial_14_field_config.png
+
+As with the configurations above, click the **Save** button to save these settings.
+
+Exploring your data
+-------------------
+
+To start exploring your data, simply click on the table name you just created in 
+the list of available tables:
+
+.. image:: _static/img/tutorial/tutorial_15_click_table_name.png
+
+By default, you’ll be presented with a Table View:
+
+.. image:: _static/img/tutorial/tutorial_16_datasource_chart_type.png
+
+Let’s walk through a basic query to get the count of all records in our table. 
+First, we’ll need to change the **Since** filter to capture the range of our data. 
+You can use simple phrases to apply these filters, like "3 years ago":
+
+.. image:: _static/img/tutorial/tutorial_17_choose_time_range.png
+
+The upper limit for time, the **Until** filter, defaults to "now", which may or may 
+not be what you want.
+
+Look for the Metrics section under the **GROUP BY** header, and start typing "Count" 
+- you’ll see a list of metrics matching what you type:
+
+.. image:: _static/img/tutorial/tutorial_18_choose_metric.png
+
+Select the *COUNT(\*)* metric, then click the green **Query** button near the top 
+of the explore:
+
+.. image:: _static/img/tutorial/tutorial_19_click_query.png
+
+You’ll see your results in the table:
+
+.. image:: _static/img/tutorial/tutorial_20_count_star_result.png
+
+Let’s group this by the *weather_description* field to get the count of records by 
+the type of weather recorded by adding it to the *Group by* section:
+
+.. image:: _static/img/tutorial/tutorial_21_group_by.png
+
+and run the query:
+
+.. image:: _static/img/tutorial/tutorial_22_group_by_result.png
+
+Let’s find a more useful data point: the top 10 times and places that recorded the 
+highest temperature in 2015.
+
+We replace *weather_description* with *latitude*, *longitude* and *measurement_date* in the

+*Group by* section:
+
+.. image:: _static/img/tutorial/tutorial_23_group_by_more_dimensions.png
+
+And replace *COUNT(\*)* with *max__measurement_flag*:
+
+.. image:: _static/img/tutorial/tutorial_24_max_metric.png
+
+The *max__measurement_flag* metric was created when we checked the box under **Max** and

+next to the *measurement_flag* field, indicating that this field was numeric and that 
+we wanted to find its maximum value when grouped by specific fields.
+
+In our case, *measurement_flag* is the value of the measurement taken, which clearly 
+depends on the type of measurement (the researchers recorded different values for 
+precipitation and temperature). Therefore, we must filter our query only on records 
+where the *weather_description* is equal to "Maximum temperature", which we do in 
+the **Filters** section at the bottom of the explore:
+
+.. image:: _static/img/tutorial/tutorial_25_max_temp_filter.png
+
+Finally, since we only care about the top 10 measurements, we limit our results to 
+10 records using the *Row limit* option under the **Options** header:
+
+.. image:: _static/img/tutorial/tutorial_26_row_limit.png
+
+We click **Query** and get the following results:
+
+.. image:: _static/img/tutorial/tutorial_27_top_10_max_temps.png
+
+In this dataset, the maximum temperature is recorded in tenths of a degree Celsius. 
+The top value of 1370, measured in the middle of Nevada, is equal to 137 C, or roughly 
+278 degrees F. It’s unlikely this value was correctly recorded. We’ve already been able

+to investigate some outliers with Superset, but this just scratches the surface of what 
+we can do.
+
+You may want to do a couple more things with this measure:
+
+* The default formatting shows values like 1.37k, which may be difficult for some 
+  users to read. It’s likely you may want to see the full, comma-separated value. 
+  You can change the formatting of any measure by editing its config (*Edit Table 
+  Config > List Sql Metric > Edit Metric > D3Format*)
+* Moreover, you may want to see the temperature measurements in plain degrees C, 
+  not tenths of a degree. Or you may want to convert the temperature to degrees 
+  Fahrenheit. You can change the SQL that gets executed agains the database, baking 
+  the logic into the measure itself (*Edit Table Config > List Sql Metric > Edit 
+  Metric > SQL Expression*)
+
+For now, though, let’s create a better visualization of these data and add it to 
+a dashboard.
+
+We change the Chart Type to "Distribution - Bar Chart":
+
+.. image:: _static/img/tutorial/tutorial_28_bar_chart.png
+
+Our filter on Maximum temperature measurements was retained, but the query and 
+formatting options are dependent on the chart type, so you’ll have to set the 
+values again:
+
+.. image:: _static/img/tutorial/tutorial_29_bar_chart_series_metrics.png
+
+You should note the extensive formatting options for this chart: the ability to 
+set axis labels, margins, ticks, etc. To make the data presentable to a broad 
+audience, you’ll want to apply many of these to slices that end up in dashboards. 
+For now, though, we run our query and get the following chart:
+
+.. image:: _static/img/tutorial/tutorial_30_bar_chart_results.png
+   :scale: 70%
+
+Creating a slice and dashboard
+------------------------------
+
+This view might be interesting to researchers, so let’s save it. In Superset, 
+a saved query is called a **Slice**. 
+
+To create a slice, click the **Save as** button near the top-left of the 
+explore:
+
+.. image:: _static/img/tutorial/tutorial_19_click_query.png
+
+A popup should appear, asking you to name the slice, and optionally add it to a 
+dashboard. Since we haven’t yet created any dashboards, we can create one and 
+immediately add our slice to it. Let’s do it:
+
+.. image:: _static/img/tutorial/tutorial_31_save_slice_to_dashboard.png
+   :scale: 70%
+
+Click Save, which will direct you back to your original query. We see that 
+our slice and dashboard were successfully created:
+
+.. image:: _static/img/tutorial/tutorial_32_save_slice_confirmation.png
+   :scale: 70%
+
+Let’s check out our new dashboard. We click on the **Dashboards** menu:
+
+.. image:: _static/img/tutorial/tutorial_33_dashboard.png
+
+and find the dashboard we just created:
+
+.. image:: _static/img/tutorial/tutorial_34_weather_dashboard.png
+
+Things seemed to have worked - our slice is here!
+
+.. image:: _static/img/tutorial/tutorial_35_slice_on_dashboard.png
+   :scale: 70%
+
+But it’s a bit smaller than we might like. Luckily, you can adjust the size 
+of slices in a dashboard by clicking, holding and dragging the bottom-right 
+corner to your desired dimensions:
+
+.. image:: _static/img/tutorial/tutorial_36_adjust_dimensions.gif
+   :scale: 120%
+
+After adjusting the size, you’ll be asked to click on the icon near the 
+top-right of the dashboard to save the new configuration.
+
+Congrats! You’ve successfully linked, analyzed, and visualized data in Superset. 
+There are a wealth of other table configuration and visualization options, so 
+please start exploring and creating slices and dashboards of your own.

http://git-wip-us.apache.org/repos/asf/incubator-superset-site/blob/886f04b0/_sources/videos.txt
----------------------------------------------------------------------
diff --git a/_sources/videos.txt b/_sources/videos.txt
new file mode 100644
index 0000000..15ef91d
--- /dev/null
+++ b/_sources/videos.txt
@@ -0,0 +1,54 @@
+Videos
+======
+
+Here is a collection of short videos showing different aspect
+of Superset.
+
+Quick Intro
+'''''''''''
+This video demonstrates how Superset works at a high level, it shows how
+to navigate through datasets and dashboards that are already available.
+
+.. youtube:: https://www.youtube.com/watch?v=3Txm_nj_R7M
+
+Dashboard Creation
+''''''''''''''''''
+This video walk you through the creation of a simple dashboard as a
+collection of data slices.
+
+- Coming soon!
+
+Dashboard Filtering
+'''''''''''''''''''
+This video shows how to create dynamic filters on dashboards, how to
+immunize certain widgets from being affected by filters.
+
+- Coming soon!
+
+Customize CSS and dashboard themes
+''''''''''''''''''''''''''''''''''
+A quick walkthrough on how to apply existing CSS templates, alter them and
+create new ones.
+
+- Coming soon!
+
+Slice Annotations
+'''''''''''''''''
+A short video on how to annotate your charts, the markdown language and
+to toggle them on dashboards.
+
+- Coming soon!
+
+Adding a Table
+''''''''''''''
+This videos shows you how to expose a new table in Superset, and how to
+define the semantics on how this can be accessed by others in the ``Explore``
+and ``Dashboard`` views.
+
+- Coming soon!
+
+Define SQL Expressions
+''''''''''''''''''''''
+A walkthrough on how to create your own derived dimensions and metrics.
+
+- Coming soon!



Mime
View raw message