cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-10103) Cloudian Connector for CloudStack
Date Wed, 25 Oct 2017 05:20:01 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-10103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16218112#comment-16218112
] 

ASF GitHub Bot commented on CLOUDSTACK-10103:
---------------------------------------------

rhtyd closed pull request #21: CLOUDSTACK-10103: Documentation for Cloudian Connector
URL: https://github.com/apache/cloudstack-docs/pull/21
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/rtd/source/_static/images/cloudian-s3_ss_cache.png b/rtd/source/_static/images/cloudian-s3_ss_cache.png
new file mode 100644
index 0000000..5d97186
Binary files /dev/null and b/rtd/source/_static/images/cloudian-s3_ss_cache.png differ
diff --git a/rtd/source/_static/images/cloudian-s3_ss_config.png b/rtd/source/_static/images/cloudian-s3_ss_config.png
new file mode 100644
index 0000000..3ad273a
Binary files /dev/null and b/rtd/source/_static/images/cloudian-s3_ss_config.png differ
diff --git a/rtd/source/_static/images/cloudian-ss_globalopt.png b/rtd/source/_static/images/cloudian-ss_globalopt.png
new file mode 100644
index 0000000..d1b9272
Binary files /dev/null and b/rtd/source/_static/images/cloudian-ss_globalopt.png differ
diff --git a/rtd/source/_static/images/cloudian-tab.png b/rtd/source/_static/images/cloudian-tab.png
new file mode 100644
index 0000000..57383e8
Binary files /dev/null and b/rtd/source/_static/images/cloudian-tab.png differ
diff --git a/rtd/source/conf.py b/rtd/source/conf.py
index f99e2ee..1fc6e29 100644
--- a/rtd/source/conf.py
+++ b/rtd/source/conf.py
@@ -63,16 +63,16 @@
 
 # General information about the project.
 project = u'Apache CloudStack'
-copyright = u'2016, Apache CloudStack'
+copyright = u'2017, Apache CloudStack'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
-version = '4.8'
+version = '4.11'
 # The full version, including alpha/beta/rc tags.
-release = '4.8.0'
+release = '4.11.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
diff --git a/rtd/source/index.rst b/rtd/source/index.rst
index 1502896..c36a7c2 100644
--- a/rtd/source/index.rst
+++ b/rtd/source/index.rst
@@ -61,6 +61,18 @@ Below you will find very specific documentation on advanced networking_
which
 you can skip if you are just getting started. Developers will also find below 
 a short developers_ guide. 
 
+
+.. _integrations:
+
+Integration Guides
+------------------
+
+.. toctree::
+   :maxdepth: 2
+
+   integration/cloudian-connector.rst
+
+
 .. _networking:
 
 
@@ -78,6 +90,7 @@ Advanced Networking Guides
    networking/ipv6
    networking/autoscale_without_netscaler.rst
 
+
 .. _developers:
 
 
diff --git a/rtd/source/integration/cloudian-connector.rst b/rtd/source/integration/cloudian-connector.rst
new file mode 100644
index 0000000..58bb170
--- /dev/null
+++ b/rtd/source/integration/cloudian-connector.rst
@@ -0,0 +1,473 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information#
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+   http://www.apache.org/licenses/LICENSE-2.0
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+
+
+The Cloudian Connector Plugin
+=============================
+
+Introduction to the Cloudian Connector Plugin
+---------------------------------------------
+
+The Cloudian (HyperStore) Connector is a native CloudStack plugin that allow
+integration between Apache CloudStack and Cloudian Management Console (CMC). The
+Connector integrates Cloudian S3 Storage into the CloudStack Management GUI and
+allows administrators to easily give their CloudStack users access to and manage
+their own S3 storage areas.
+
+Compatibilty
+~~~~~~~~~~~~
+
+The following table shows the compatiblity of Cloudian Connector with CloudStack.
+
+.. cssclass:: table-striped table-bordered table-hover
+
++---------------------+----------------------+-------------------------+
+| Connector Version   | CloudStack version   | Cloudian Compatibility  |
++=====================+======================+=========================+
+| 4.9_6.2-1           | 4.9                  | Version 6.2 and onwards |
++---------------------+----------------------+-------------------------+
+| 4.11+               | 4.11+                | Version 6.2 and onwards |
++---------------------+----------------------+-------------------------+
+
+Table: Support Matrix
+
+.. note::
+   Starting CloudStack 4.11, the Connector will be part of the CloudStack
+   release and will not need to be externally installed.
+
+Connector Overview
+------------------
+
+Single-Sign-On Integration
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The connector plugin adds a Cloudian Storage button to the CloudStack UI. This
+button is available for all users on the bottom left of the menu.
+
+.. figure:: /_static/images/cloudian-tab.png
+   :align: center
+   :alt: a screenshot of an enabled Cloudian Connector
+
+When a user clicks this button, a new window or tab (depending on the web
+browser preferences) is opened for the HyperStore CMC GUI. The CloudStack user
+is automatically logged in to CMC as the correctly mapped HyperStore user using
+Single-Sign-On (SSO).
+
+With the connector enabled, when the user clicks ‘Log Out’ in the CloudStack UI
+this first logs the user out of CloudStack and then redirects the page to log
+out any logged-in Cloudian user out of CMC GUI and finally redirects the page to
+the CloudStack login page.
+
+Single-Sign-On is a technique where CloudStack and HyperStore are configured to
+trust each other. This is achieved by configuring both HyperStore and the
+CloudStack connector with the same SSO Shared Key. The CloudStack connector
+creates a special login URL for CMC which it signs using this shared key. Upon
+receiving the special signed login URL, CMC validates the request by comparing
+the signature to its own copy of the shared key and the user is automatically
+logged in.
+
+User Mapping and Provisioning/De-provisioning
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+CloudStack domains are mapped to Cloudian Groups. CloudStack accounts within
+those domains are mapped to Cloudian users. The Cloudian user and group are
+created on demand if it doesn’t already exist when the CloudStack user accesses
+CMC through the Cloudian Storage button. When accounts and domains are created
+or removed in CloudStack, they automatically create or remove users or groups in
+CMC.
+
+.. cssclass:: table-striped table-bordered table-hover
+
++---------------------+----------------------------+
+| CloudStack Entity   | Equivalent Cloudian Entity |
++=====================+============================+
+| Account             | User                       |
++---------------------+----------------------------+
+| Domain              | Group                      |
++---------------------+----------------------------+
+
+Table: Mapping Between Cloudian and CloudStack
+
+.. note::
+   Adding groups or users directly through Cloudian does not add
+   corresponding CloudStack Domains or Accounts. The integration is driven
+   completely from the CloudStack side.
+
+Special Admin User Mapping
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The special CloudStack admin is are mapped to a special HyperStore Admin user
+account which defaults to the user id admin. As the admin user on HyperStore is
+configurable, there is a configuration option to control this mapping. This
+mapping dictates which HyperStore user is automatically logged in using SSO when
+the CloudStack admin user clicks "Cloudian Storage".
+
+.. note::
+   The Cloudian Admin user default is called admin. Older versions of
+   Cloudian used to use admin@cloudian.com.
+
+DNS Resolution Requirements
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The CloudStack Management Server will need to be access the Cloudian admin
+service. The Cloudian admin service is commonly run on the same nodes as your
+Cloudian S3 servers. The admin service is used to provision and deprovision
+Cloudian users and groups automatically by CloudStack.
+
+Additionally, your CloudStack users will need to be able to resolve your CMC
+server hostname on their desktops so that they can access CMC.
+
+Example domain names that should resolve:
+
+.. cssclass:: table-striped table-bordered table-hover
+
++---------------------+----------------------+------------------------------+
+| Resolvable Name     | Required By          | Description                  |
++=====================+======================+==============================+
+| mgmt.abc-cloud.com  | User's browser       | CloudStack Management Server |
++---------------------+----------------------+------------------------------+
+| cmc.abc-cloud.com   | User's browser       | Cloudian CMC                 |
++---------------------+----------------------+------------------------------+
+| admin.abc-cloud.com | Management Server    | Cloudian Admin Server        |
++---------------------+----------------------+------------------------------+
+
+Table: DNS Name Resolution Example
+
+
+Configuring the Cloudian Connector
+----------------------------------
+
+Prerequisites
+~~~~~~~~~~~~~
+
+Cloudian ships with SSO disabled by default. You will need to enable it on each
+CMC server. Additionally, you will need to choose a unique SSO shared key that
+you will also configure in the CloudStack connector further below.
+
+Edit Puppet config to enable SSO on all CMC servers:
+
+   ::
+
+     # vi /etc/cloudian-[version]-puppet/modules/cmc/templates/mts-ui.properties.erb
+       sso.enabled=true
+       sso.shared.key=YourSecretKeyHere
+
+
+.. note::
+   Once configured in Puppet, you should roll out out to each CMC server and
+   restart CMC services. Please refer to the HyperStore documentation for how to
+   do this.
+
+Connector Configuration
+~~~~~~~~~~~~~~~~~~~~~~~
+
+The main way to configure, enable and disable the connector is using the
+CloudStack global setting. The global settings provide an easy way to configure
+the connector and synchronize setting across multiple management server(s). The
+following global setting can be accessed and changed using the CloudStack UI:
+
+.. cssclass:: table-striped table-bordered table-hover
+
++------------------------------+------------------------------------------------+
+| Global Setting               | Description                                    |
++==============================+================================================+
+| cloudian.connector.enabled   | Setting to enable/disable the plugin           |
++------------------------------+------------------------------------------------+
+| cloudian.admin.host          | The Cloudian admin server host                 |
++------------------------------+------------------------------------------------+
+| cloudian.admin.port          | The Cloudian admin server port, usually 19443  |
+|                              | (https) or 18081 (http)                        |
++------------------------------+------------------------------------------------+
+| cloudian.admin.protocol      | The Cloudian admin server protocol, http/https |
++------------------------------+------------------------------------------------+
+| cloudian.validate.ssl        | Whether to validate SSL certificate of Cloudian|
+|                              | admin service while making API calls           |
++------------------------------+------------------------------------------------+
+| cloudian.admin.user          | Basic auth user name for Cloudian admin server |
++------------------------------+------------------------------------------------+
+| cloudian.admin.password      | Basic auth password for Cloudian admin server  |
++------------------------------+------------------------------------------------+
+| cloudian.api.request.timeout | The admin API request timeout in seconds       |
++------------------------------+------------------------------------------------+
+| cloudian.cmc.admin.user      | The user id of the CMC admin that maps to      |
+|                              | CloudStack admin user                          |
++------------------------------+------------------------------------------------+
+| cloudian.cmc.host            | The Cloudian Management Console hostname       |
++------------------------------+------------------------------------------------+
+| cloudian.cmc.port            | The Cloudian Management Console port           |
++------------------------------+------------------------------------------------+
+| cloudian.cmc.protocol        | The Cloudian Management Console protocol       |
++------------------------------+------------------------------------------------+
+| cloudian.sso.key             | The shared secret as configured in Cloudian CMC|
++------------------------------+------------------------------------------------+
+
+Table: Cloudian Connector Global Settings
+
+.. note::
+   Change in only ‘cloudian.connector.enabled’ setting requires restarting of
+   all the CloudStack management server(s), rest of the setting can be changed
+   dynamically without requiring to restart the CloudStack management server(s).
+
+Enabling the Cloudian Connector
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Cloudian Connector comes disabled by default, enabling the connector is the
+last step. You should have already configured the Cloudian Connector global
+settings. To enable the connector, ensure that the global setting
+"cloudian.connector.enabled" is set to true. Finally, restart each of the
+management server(s) to reload and enable the connector.
+
+For example, here is how you can restart the CloudStack management server
+installed on CentOS7:
+
+   ::
+
+     # systemctl restart cloudstack-management
+
+
+Troubleshooting
+~~~~~~~~~~~~~~~~
+
+Most of the trouble you may run into will be configuration related.
+
+There are a few things which can go wrong for SSO. Here are the most common
+problems and things to check:
+
+-  Does the global settings cloudian.cmc.admin.user point to the correct Cloudian
+   (admin) user?
+
+-  Is SSO configured and enabled on Cloudian HyperStore CMC?
+
+-  Check for errors in the CMC log file.
+
+-  Are both CloudStack and HyperStore CMC configured with the same cloudian.sso.key?
+
+-  Check the /var/log/cloudstack/management/management-server.log file and
+   search for errors relating to SSO.
+
+-  Try access the CMC host directly from the problem users host using the
+   configured cloudian.cmc.host, cloudian.cmc.port and cloudian.cmc.protocol
+   configured in the CloudStack global settings.
+
+-  If you log out of the management server and log in again, does the Cloudian
+   Storage button work?
+
+
+Adding/Deleting Domains or Accounts fails: These operations use the Cloudian
+Admin Server. It's likely that something has changed with the connection or the
+admin server is down. Check list:
+
+-  Is the admin server alive and listening?
+
+-  Try access the admin server host directly from the problem users host using
+   the configured cloudian.admin.host, cloudian.admin.port and
+   cloudian.admin.protocol configured in the CloudStack global settings. Check the
+   configured auth settings cloudian.admin.user and cloudian.admin.password.
+
+-  If you’re experiencing timeout issues, trying changing the API timeout value
+   defined in cloudian.api.request.timeout global setting.
+
+-  Look for errors in the admin log file /var/log/cloudian/cloudian-admin.log.
+
+
+------------
+
+
+Cloudian as CloudStack Secondary Storage
+----------------------------------------
+
+This section is a supplementary guide for CloudStack and describes how to
+configure CloudStack to use Cloudian HyperStore as Secondary Storage. Please
+also review CloudStack’s documentation (Getting Started Guide) for configuring
+and using S3 as Secondary Storage.
+
+CloudStack, as of version 4.2.1, can utilize Cloudian HyperStore as S3 Secondary
+Storage out of the box. There is no need for any modifications or to install any
+connectors. Secondary Storage is used to store ISOs, Templates, Snapshots and
+Volumes.
+
+.. note::
+   CloudStack still requires an NFS Secondary Storage Staging Server with is
+   mentioned in the requirements below.
+
+Requirements:
+
+-  CloudStack 4.5+ (installed/configured and running)
+-  Cloudian HyperStore 5.0 or greater (installed/configured and running)
+
+NFS Secondary Storage Staging Server Requirement
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The use of S3 as Secondary Storage for CloudStack also requires an NFS server.
+The NFS server is required because the various hypervisors cannot yet talk
+directly to S3 but instead talk through the standard file system API. As such,
+CloudStack requires an NFS staging server which the Hypervisors use to read and
+write data from/to. The NFS storage requirements for the staging server are
+small however as space is only required while objects are staged (moving)
+between the S3 server and the VMs.
+
+
+DNS Name Resolution Requirement
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+All CloudStack Management Servers, system VMs and customer VMs (if required)
+must be able to resolve your S3 bucket names. Usually, if you already have
+Cloudian installed and running in your environment, this is already working.
+At a minimum the following names should resolve to the correct IP addresses
+using the DNS server that your Management Server and System VMs are using.
+
+.. cssclass:: table-striped table-bordered table-hover
+
++---------------------+------------------------------+
+| Example Name        | DNS Name Types               |
++=====================+==============================+
+| s3.mycloud.com      | Cloudian S3 Endpoint         |
++---------------------+------------------------------+
+| sec.s3.mycloud.com  | Bucket for Secondary Storage |
++---------------------+------------------------------+
+| s3-admin.mycloud.com| Cloudian Admin Server        |
++---------------------+------------------------------+
+
+Table: Example Domain Names that should Resolve on CloudStack Servers
+
+Adding Cloudian as CloudStack Secondary Storage
+-----------------------------------------------
+
+Setup a Cloudian User and Bucket for Secondary Storage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+S3 Secondary Storage stores the CloudStack templates, snapshots etc in a
+dedicated S3 Bucket. To properly configure CloudStack you will need to know the
+S3 Bucket name and how to access your S3 Server (the S3 endpoint, access key and
+secret key).
+
+Below, we will create a dedicated Cloudian user and a dedicated bucket which we
+will assign for use as Secondary Storage.
+
+Create a dedicated user/group:
+
+-  Login to the Cloudian Management Console (CMC) as the Cloudian admin user.
+
+-  Create a new group called cloudstack. Any group name is ok.
+
+-  Create a new user called cloudstack in the cloudstack group. Any user name is ok.
+
+
+Create a dedicated bucket:
+
+-  Login to CMC as the cloudstack user created above.
+
+-  Create a bucket called secondary. Any bucket name will do.
+
+-  On the top menu bar on the right hand side, use the drop down menu under your
+   user name to select Security Credentials and copy and paste your Access and
+   Secret Keys to a note for later use. CloudStack will need these when you attach
+   Cloudian as Secondary Storage in a later step below.
+
+Open Up Access to your S3 Network from Secondary Storage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If your S3 server is on a different network to your Secondary Storage VM, you
+will need to open up access to the S3 network. This also allows users to
+download templates from their S3 object store areas.
+
+.. figure:: /_static/images/cloudian-ss_globalopt.png
+   :align: center
+   :alt: a screenshot of changing global settings
+
+.. note::
+   Editing the Global Settings requires you to restart the management server(s).
+
+Add an NFS Secondary Storage Staging Server
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As mentioned previously, S3 Secondary Storage currently requires the use of an
+NFS Secondary Staging Server. Add NFS Secondary Storage Staging Server:
+
+-  Login to CloudStack Management Server as the admin user.
+
+-  Navigate to Infrastructure → Secondary Storage.
+
+-  Click Select View and select Secondary Staging Store.
+
+-  Click Add Secondary Staging Store.
+
+-  Configure the zone, server and path for your desired secondary staging store.
+   For example nfs.mycloud.com and /export/staging.
+
+.. figure:: /_static/images/cloudian-s3_ss_cache.png
+   :align: center
+   :alt: a screenshot of adding secondary staging store
+
+Attach Cloudian as Secondary Storage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+CloudStack supports using either S3 or NFS as Secondary Storage but not both.
+The below instructions assume you are not using Secondary Storage on NFS and
+that you can delete it to add the S3 storage.
+
+.. note::
+  Already using NFS for Secondary Storage with CloudStack? You need to migrate
+  your Secondary Storage. Refer to CloudStack’s instructions for migrating
+  existing NFS Secondary Storage to an S3 object storage. CloudStack 4.5 onwards
+  supports migrating data via special commands which are described in the Getting
+  Started Guide in a section titled Upgrading from NFS to Object Storage.
+
+Adding S3 Secondary Storage:
+
+-  Login to CloudStack Management Server as the admin user.
+
+-  Navigate to Infrastructure → Secondary Storage.
+
+-  If it exists, select and delete any existing NFS Secondary Storage server
+   setting. NOTE: Do not do this if you want to migrate existing NFS secondary
+   storage to S3. Instead, see warning above.
+
+-  Click the Add Secondary Storage button. This will open up a pop-up form which
+   you can fill out similarly to below.
+
+.. figure:: /_static/images/cloudian-s3_ss_config.png
+   :align: center
+   :alt: a screenshot of configuring S3 secondary storage
+
+.. note::
+   CloudStack doesn’t currently allow you to re-edit the S3 configuration so take
+   time to double check what you enter. If you make a mistake the only options
+   currently are either a) delete and recreate the storage or b) directly edit the
+   entry in the database.
+
+When you have finished adding Cloudian as Secondary Storage in the previous
+steps, CloudStack will populate the new secondary storage with the system and
+default templates. This can take some time do download as the templates are
+quite big.
+
+.. note::
+   You can check if the system template and the default template have properly
+   downloaded to the new secondary storage by navigating to Templates, selecting a
+   template, clicking on the Zones tab and checking its Status is Ready 100%
+   Downloaded.
+
+.. note::
+   Should you continue to have problems, sometimes it is necessary to restart
+   the Secondary Storage VM. You can do this by navigating to Infrastructure,
+   System VMs, selecting and rebooting the Secondary Storage VM.
+
+CloudStack should now ready to use Cloudian HyperStore for S3 Secondary Storage.
+
+Revision History
+----------------
+
+-  Fri Oct 6 2017 Rohit Yadav rohit.yadav@shapeblue.com Documentation
+   created for 4.11.0 version of the Cloudian Connector Plugin


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Cloudian Connector for CloudStack
> ---------------------------------
>
>                 Key: CLOUDSTACK-10103
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-10103
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>            Reporter: Rohit Yadav
>            Assignee: Rohit Yadav
>             Fix For: Future, 4.11.0.0
>
>
> Cloudian Connector for CloudStack that supports following:
> - UI based SSO into Cloudian CMC from CloudStack
> - Auto-provisioning and auto-deprovisioning of CloudStack accounts and domains as Cloudian
Users and Groups, when plugin is enabled
> - Easy configuration using CloudStack global settings
> - ASF 2.0 license compatible



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message