Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 789A0200C5D for ; Fri, 7 Apr 2017 16:58:10 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 772CC160BA2; Fri, 7 Apr 2017 14:58:10 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2FBE6160B97 for ; Fri, 7 Apr 2017 16:58:08 +0200 (CEST) Received: (qmail 72232 invoked by uid 500); 7 Apr 2017 14:58:07 -0000 Mailing-List: contact commits-help@airavata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airavata.apache.org Delivered-To: mailing list commits@airavata.apache.org Received: (qmail 72208 invoked by uid 99); 7 Apr 2017 14:58:06 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Apr 2017 14:58:06 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 51FAAE0F98; Fri, 7 Apr 2017 14:58:06 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: scnakandala@apache.org To: commits@airavata.apache.org Date: Fri, 07 Apr 2017 14:58:07 -0000 Message-Id: <01642c7e02e14ab88891067627ad3723@git.apache.org> In-Reply-To: <29941f2453aa42f6bba00ac917a1342f@git.apache.org> References: <29941f2453aa42f6bba00ac917a1342f@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/2] airavata-docs git commit: backing up 0.16 release documents archived-at: Fri, 07 Apr 2017 14:58:10 -0000 backing up 0.16 release documents Project: http://git-wip-us.apache.org/repos/asf/airavata-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/airavata-docs/commit/9259dfe4 Tree: http://git-wip-us.apache.org/repos/asf/airavata-docs/tree/9259dfe4 Diff: http://git-wip-us.apache.org/repos/asf/airavata-docs/diff/9259dfe4 Branch: refs/heads/master Commit: 9259dfe44e1bf8511631d215b00275ad75bbf09e Parents: 2cdb3e6 Author: scnakandala Authored: Fri Apr 7 10:58:01 2017 -0400 Committer: scnakandala Committed: Fri Apr 7 10:58:01 2017 -0400 ---------------------------------------------------------------------- 0.16/Admin-Maintenance.md | 55 +++++ 0.16/Admin-Tutorials.md | 198 ++++++++++++++++++ 0.16/Airavata-Installation.md | 134 ++++++++++++ 0.16/Airavata-Upgrades.md | 41 ++++ 0.16/AiravataApi.md | 77 +++++++ 0.16/Application-CookBook.md | 43 ++++ 0.16/CPI.md | 291 ++++++++++++++++++++++++++ 0.16/Contact-Us.md | 14 ++ 0.16/FAQ.md | 220 ++++++++++++++++++++ 0.16/Gateway-Configurations.md | 124 ++++++++++++ 0.16/Models.md | 292 +++++++++++++++++++++++++++ 0.16/Own-WSO2-IS-Configuration.md | 73 +++++++ 0.16/PGA-Installation.md | 197 ++++++++++++++++++ 0.16/PGA-Upgrades.md | 9 + 0.16/Sharing-Docs.md | 79 ++++++++ 0.16/User-Profiles.md | 47 +++++ 0.16/User-Tutorials.md | 6 + 0.16/WSO2-IS-Configuration.md | 42 ++++ 0.16/airavata-properties.md | 66 ++++++ 0.16/css/extra.css | 29 +++ 0.16/img/AppCatalog.png | Bin 0 -> 102779 bytes 0.16/img/AppDeployment.png | Bin 0 -> 86972 bytes 0.16/img/AppInterface1.png | Bin 0 -> 104215 bytes 0.16/img/AppInterface2.png | Bin 0 -> 167809 bytes 0.16/img/AppModule.png | Bin 0 -> 87288 bytes 0.16/img/GauAppDep.png | Bin 0 -> 232179 bytes 0.16/img/GauAppInt1.png | Bin 0 -> 226844 bytes 0.16/img/GauAppInt2.png | Bin 0 -> 212219 bytes 0.16/img/GauAppInt3.png | Bin 0 -> 106055 bytes 0.16/img/MultipleWaystoUseAiravata.png | Bin 0 -> 72711 bytes 0.16/img/MultipleWaystoUseAiravata1.png | Bin 0 -> 129098 bytes 0.16/img/PGAMenu&SubMenu.png | Bin 0 -> 102694 bytes 0.16/img/UserProfiles.png | Bin 0 -> 78923 bytes 0.16/img/UserProfilesOLD.png | Bin 0 -> 101847 bytes 0.16/index.md | 24 +++ 0.16/mkdocs.yml | 38 ++++ 36 files changed, 2099 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/airavata-docs/blob/9259dfe4/0.16/Admin-Maintenance.md ---------------------------------------------------------------------- diff --git a/0.16/Admin-Maintenance.md b/0.16/Admin-Maintenance.md new file mode 100644 index 0000000..b40faef --- /dev/null +++ b/0.16/Admin-Maintenance.md @@ -0,0 +1,55 @@ +## Gateway Maintenance + +###

Assist Gateway Users

+Q1. One of my gateway users wants me to investigate his FAILED experiment. How should I proceed? +
Answer: To investigate a specific FAILED experiment or failed experiment within a time frame,
+1. Navigate to AdminDashboard ⇒ Experiment Statistics
+2. Search the experiment by using the experiment ID or by giving the Creation Time 'To' and 'From'
+3. You could also use pre-defined selection criteria ('Get Experiments from Last 24 Hours' OR 'Get Experiments from Last Week')
+ + +###

Day-to-Day Maintenance

+Q2. One of the resources used by my gateway is not available for the day. How to stop user job submissions? +
Answer: To temporarily stop users submitting jobs to a particular resource...
+1. Navigate to AdminDashboard ⇒ Compute Resources (Browse)
+2. Un-check the 'Enabled' box for the specific resource. This disables job submissions for the resource
+3. To enable job submission simply check the box and you are back in track!
+ +Q3. How to upgrade access for a gateway user? +
Answer: User access can be upgraded or downgraded by changing the assigned user role.
+1. Navigate to Admin Dashboard ⇒ Users (Browse)
+2. Search for the specific user and click 'Check All Roles'
+3. From this you can remove/add roles.
+4. For each modification user will receive an email with 'Your Privileges has Changed!'
+ +Q4. Gateway users who has role 'gateway-user' cannot access experiment creation. Seems like privileges are not accessible.! +
Answer: User roles are given through PGA Admin Dashboard. If the roles are set correctly then check pga_config.php
+1. To check the config file use +
vi /var/www/html/airavata-php-gateway/app/config/pga_config.php
+2. Roles attached to users should exists in the config file against correct role attribute type +

+        /**
+         * Admin Role Name
+         */
+        'admin-role-name' => 'admin',
+        /**
+         * Read only Admin Role Name
+         */
+        'read-only-admin-role-name' => 'admin-read-only',
+        /**
+         * Gateway user role
+         */
+        'user-role-name' => 'airavata-user',
+
+ +Q5. I have ran out of allocation for my current community account used in gateway for a resource. What should I do? +
Answer:
+1. If you have another community account you could update the information in Admin Dashboard ⇒ Gateway Profile under Compute Resource Preferences.
+2. Select the resource and modify account information and save.
+3. Login to the resource as the new user and update the authorized_keys with the public key assigned to the resource.
+ +Q6. I want to send notices to my gateway users. How? +
Answer:
+1. Navigate to Admin Dashboard⇒Notices
+2. Use 'Create a New Notice' button add a new notice.
+3. Based on the published date you provided it will be available for users as a Notice ! http://git-wip-us.apache.org/repos/asf/airavata-docs/blob/9259dfe4/0.16/Admin-Tutorials.md ---------------------------------------------------------------------- diff --git a/0.16/Admin-Tutorials.md b/0.16/Admin-Tutorials.md new file mode 100644 index 0000000..1292f2f --- /dev/null +++ b/0.16/Admin-Tutorials.md @@ -0,0 +1,198 @@ +## Admin Dashboard Configurations + +This page is dedicated to Gateway Admins! + +### Prior to starting your configurations +1. You have admin access to the PGA portal, Admin Dashboard + - How to check? + - Log in to the gateway portal + - You would land on Admin Dashboard with "Let's Get Started!" + - Click 'Browse Application Modules' + - Do you see button 'Create new Application Interface' ? Then you are good. +2. Storage resource ID added to the pga_config.php
+ - If not go to + - Admin Dashboard --> Storage Resources --> Browse + - Locate your resource and copy the ID + - Paste it in pga_config.php file in pga/app/config directory. +3. Credentials generated, assigned and added to authorized_keys files of resources + - To generate key refer Credential Store + - Token Assignment Gateway Management + - Copy the public key for the assigned token frm credential store and add it to authorized_keys file in both compute and storage resources. + +NOTE: If you are using a hosted gateway the 2 and 4 would be taken cared by the SciGaP team. + +### Select Your Quick Start Tutorial +1. Running Echo on Local Machine
+2. Gaussian Job Submission to Comet (XSEDE resource)
+3. Add Pre and Post Job Commands for an Application
+4. Add Environment Variables for an Application Deployment
+5. Know your Compute & Storage Resources
+6. Gateway Preference Management
+7. Managing User Accounts
+8. Communication with Gateway Users
+9. Monitor Gateway Traffic
+10. Using Credential Store
+ + +#####
Running Echo on Local Machine
+Quickest way to confirm Airavata and PGA setup. This will tell you what you need to do to Echo a simple 'Hello World' in your local machine + +1. Create new application module: Echo + - Navigate to Admin Dashboard → App Catalog → Apllication Module + - Click Create a new Application Module + - Enter Application Module Name: Echo + - Enter Application Module Version: Echo 1.0 (Not mandatory) + - Enter Description: Echo application for testing + - Create + - This create the Echo module
+![Screenshot](img/AppModule.png)

+2. Create the application interface: Echo + - Navigate to Admin Dashboard → App Catalog → Application Interface + - Click 'Create new Application Interface' + - Add Application Name: Echo + - Add Application Description: Echo Interface for testing + - Select Application Module: Echo + - Set 'Enable Archiving Working Directory' to False (Why? - This is set to true if you want to bring back all the files in working directory back to PGA) + - Set 'Enable Optional File Inputs' to False (Why? - Set to false because there won't be any additional optional inputs for Echo) + - Provide application inputs + - Click Add Application Input + - Name: Input-to-Echo + - Value: Echo Test 1......2......3....... (This value can be overridden at experiment creation) + - Type: STRING + - Application Arguments: + - Standard Input: False (Why? - Futuristic property and not in real use at the moment) + - User Friendly Description: Enter STRING input for Echo (Not mandatory) + - Input Order: 1 + - Data is Staged: True + - Is the Input Required: True + - Required in Commandline: True + - Meta Data: + - Provide application outputs + NOTE: 3 application outputs to define.
+ - 1st output + - Click Add Application Output + - Name: Echo-Standard-Out + - Value: + - Type: STDOUT + - Application Argument: + - Data Movement: False + - Is the Output required?: True + - Required on command line?: True + - Location: + - Search Query:
+ - 2nd output + - Click Add Application Output + - Name: Echo-Standard-Error + - Value: + - Type: STDERR + - Application Argument: + - Data Movement: False + - Is the Output required?: True + - Required on command line?: True + - Location: + - Search Query:
+![Screenshot](img/AppInterface1.png) ![Screenshot](img/AppInterface2.png)

+3. Create the application deployment: Echo on Local Machine + - Navigate to Admin Dashboard → App Catalog → Application Deployment + - Click 'Create new Application Deployment' + - Application Module: Echo + - Application Compute Host: Local (Local machine has to be added as a compute resource prior to this step) + - Application Executable Path: /home/airavata/ECHO/echo_wrapper.sh (Local to where you have airavata installed) + - Application Parallelism Type: SERIAL
+![Screenshot](img/AppDeployment.png) +4. Echo_wrapper.sh contains; +

+    #!/bin/bash
+    #sleep 10
+    echo "Echoed_Output=$*" 
+
+ +#####
Gaussian Job Submission to Comet (XSEDE resource)
+This is a tutorial to configuring and running an application on XSEDE resource through PGA portal. + +1. Create new application module: Gaussian + - Navigate to Admin Dashboard → App Catalog → Apllication Module + - Click Create a new Application Module + - Enter Application Module Name: Gaussian + - Enter Application Module Version: Gaussian 09 (Not mandatory) + - Enter Description: Gaussian provides capabilities for electronic structure modeling + - Create + - This create the Gaussian module
+2. Create the application interface: Gaussian + - Navigate to Admin Dashboard → App Catalog → Application Interface + - Click 'Create new Application Interface' + - Add Application Name: Gaussian + - Add Application Description: Gaussian provides capabilities for electronic structure modeling + - Select Application Module: Gaussian ( If multiple modules are using same interface here you can add multiple modules. Ex: Gaussian 03 and Gaussian 09 modules) + - Set 'Enable Archiving Working Directory' to True (Why? - This is set to true when you want to bring back all the files in working directory back to PGA. Caution: If there are very large files they may not be able to SCP) + - Set 'Enable Optional File Inputs' to False (Why? - Set to false because there won't be any additional optional inputs for Gaussian) + - Provide application inputs + - Click Add Application Input + - Name: Input-File + - Value: + - Type: URI (Why - This is the type for file uploads) + - Application Arguments: + - Standard Input: False (Why? - Futuristic property and not in real use at the moment) + - User Friendly Description: Gaussian input file specifying desired calculation type, model chemistry, molecular system and other parameters. (This is information to the user at creating job experiment. Not mandatory) + - Input Order: 1 + - Data is Staged: False + - Is the Input Required: True + - Required in Commandline: True + - Meta Data: + - Provide application outputs
+ NOTE: 3 application outputs to define.
+ - 1st Output + - Click Add Application Output + - Name: Gaussian-Application-Output + - Value: Gaussian.log + - Type: URI + - Application Argument: + - Data Movement: True + - Is the Output required?: True + - Required on command line?: True + - Location: + - Search Query:
+ - 2nd output + - Click Add Application Output + - Name: Gaussian-Standard-Out + - Value: + - Type: STDOUT + - Application Argument: + - Data Movement: False + - Is the Output required?: True + - Required on command line?: True + - Location: + - Search Query:
+ - 3rd output + - Click Add Application Output + - Name: Gaussian-Standard-Error + - Value: + - Type: STDERR + - Application Argument: + - Data Movement: False + - Is the Output required?: True + - Required on command line?: True + - Location: + - Search Query:
+![Screenshot](img/GauAppInt1.png) ![Screenshot](img/GauAppInt2.png) ![Screenshot](img/GauAppInt3.png)

+3. Create the application deployment: Gaussian on Comet + - Navigate to Admin Dashboard → App Catalog → Application Deployment + - Click 'Create new Application Deployment' + - Application Module: Gaussian + - Application Compute Host: comet.sdsc.edu (Your comet specific gateway preferences need to be added in 'Gateway Management' prior to this) + - Application Executable Path: g09 + - Application Parallelism Type: SERIAL + - Application Deployment Description: Gaussian provides capabilities for electronic structure modeling. + - Module Load Commands: module load gaussian + - Post Job Commands: mkdir -p "$PWD"_restart; cp *.chk ${pwd}_restart; mv *.rwf* ${pwd}_restart; (Why? - any command you would want to execute after post job execution)
+![Screenshot](img/GauAppDep.png) + +#####
Add Pre and Post Job Commands for an Application
+This tutorial demonstrates use of pre and post job commands and how they appear in job submission script + +1. + +Work-in-Progress + +Refer Application Configuration for more generic details. + http://git-wip-us.apache.org/repos/asf/airavata-docs/blob/9259dfe4/0.16/Airavata-Installation.md ---------------------------------------------------------------------- diff --git a/0.16/Airavata-Installation.md b/0.16/Airavata-Installation.md new file mode 100644 index 0000000..11ebb7b --- /dev/null +++ b/0.16/Airavata-Installation.md @@ -0,0 +1,134 @@ +##

Install Apache Airavata

+ +Select...
+[](#Airavata)   +[](#AiravataCent)
+ +###

General Prerequisites

+1. JAVA 8 + - Java installation on CentOS, Mac, Windows, etc.. - Oracle JAVA Installation +2. RabbitMQ + - Download the RabbitMQ Binary
Select the download file as per the operating system of your machine/server. + - Same link has installing guide documentation as well. E.g.:MAC Installation Guide + - For detailed information on getting RabbitMQ started, stopped, etc please visit RabbitMQ Documentation +3. Maven + - Download Maven (java based code building tool). +4. MySQL Database + +###

Airavata Installation on CentOS 7

+NOTE: Airavata installation on other operating systems are similar with minor changes.
+#### Pre-Installations +1. CentOS 7 Default open JDK 1.8.0. (minimum) is sufficient. Use "yum search jdk" to find available versions. Verify versions using "javac -version". Be sure to install the developer version. +
yum install java-1.8.0-openjdk-devel.x86_64
+2. Download RabbitMQ binary for CentOS 7 +Download RabbitMQ Binary for CentOS
+3. Prerequisite for RabbitmQ Erlang can be installed using yum +
yum install erlang
+4. Unzip the downloaded RabbitMQ tar file into a folder in your local machine. +
tar -xvf rabbitmq-server-mac-standalone-3.4.1.tar.gz
+5. Start the RabbitMQ server in the bin folder using; + 
./sbin/rabbitmq-server start
+6. Install Maven using yum install. ((install the latest maven 3.3.9. Default Maven default of centOS 7). +
yum install maven
+7. Install MySQL database +
yum install mariadb-server
+8. Start maria DB with; +
systemctl start mariadb
+9. While maria DB is running run
+
mysql_secure_installation

+When executing above it will ask you for root password; provide it.
+10. Now login as the root user providing the password you gave above.
+10. Create databases required for Airavata
+
create database app_catalog;

+
create database experiment_catalog;

+
create database replica_catalog;

+
create database credential_store;

+
create database workflow_catalog;

+11. Grant permission to these databases for a new user which would be used by Airavata

+Command syntax: 
+
 grant all privileges on <DB-Name>.⁎ to '<username>'@'%' identified by '<password>'; 

+E.g.: 
+
grant all privileges on app_catalog.⁎ to 'airavata'@'%' identified by 'airavata';

+

+NOTE: Grant permission to every databased created above. % can be replaced by  'localhost' (if DB is also in the same server as airavata). If DB is in a different server give the server name.
+

+
+#### Airavata Installation
+1. Create a folder in your local machine (E.g.: mkdir LocalAiravata).

+2. Clone the master source (If you have not taken a clone prior) code from github to the created folder.

+
git clone https://github.com/apache/airavata.git

+3. After cloning is completed, build the source code by executing following maven command (In LocalAiravata/airavata);
+
mvn clean install

+Hint: To avoid tests (recommended for first time users) use
+
mvn clean install -Dmaven.test.skip=true

+4. Locate the tar file in target directory
+Path:
+
cd airavata/distribution/target/

+5. Navigate to locally created directory (LocalAiravata) copy the tar file
+
cp airavata/distribution/target/apache-airavata-server-0.16-SNAPSHOT-bin.tar.gz ./

+OR
+
cp airavata/distribution/target/apache-airavata-server-0.16-SNAPSHOT-bin.zip ./

+6. Now unzip either the tar or zip file of Airavata server distribution;
+
unzip apache-airavata-server-0.16-SNAPSHOT-bin.zip

+OR
+
tar xvzf apache-airavata-server-0.16-SNAPSHOT-bin.tar.gz

+7. Generate Credential store keystore file in the created local directory.
+
	keytool -genseckey -alias airavata -keyalg AES -keysize 128 -storetype jceks -keystore airavata_sym.jks

+For more information visit Credential Store Configuration Documentation
+8. In order to copy mysql jar file (always download the latest available); navigate to Airavata lib directory
+
cd /LocalFolderPath/apache-airavata-server-0.16-SNAPSHOT/lib

+Now copy the mysql jar (e.g. mysql-connector-java-5.1.39-bin.jar) to lib  directory (Download mysql.jar).
+9. Navigate to bin folder which contains file airavata-server.properties and open it;
+
vi apache-airavata-server-0.16-SNAPSHOT/bin/airavata-server.properties

+10. Update relevant necessary properties in airavata-server.properties file.

+Change as required. For more details refer; Airavata Property File.
+	- In sections; API Server Registry Configuration, Application Catalog DB Configuration, Data Catalog DB Configuration, Workflow Catalog DB Configuration, Credential Store Module Configuration

+		- Comment out derby DB properties.

+		- Change MySQL configurations as per the databases created above.
+	- Make sure gateway ID is properly defined;
 
+		default.registry.gateway=php_reference_gateway
+	- Server module Configuration
+		- Make sure all servers required to start are added as given 

+			servers=apiserver,orchestrator,gfac,credentialstore
+	- API Server SSL Configurations
+		- Give the correct path for key generation file. This is in the bin directory and it is shipped default with Airavata.

+		apiserver.keystore=/home/airavata/LocalAiravata/apache-airavata-server-0.16-SNAPSHOT/bin/airavata.jks
+	- Credential Store module Configuration
+		- Add the path to SSH key generation file 

+		E.g.: credential.store.keystore.url=/home/airavata/LocalAiravata/airavata-sym.jks
+	-  API Security Configuration
+		- Make sure 

+		api.secured=false

+		TLS.enabled=false
+	-  Monitoring Module Configuration
+      	- Add your email address, username and password for email monitoring.
 This is the email account the job status change emails will be received from compute resources.
+		 email.based.monitor.host=imap.gmail.com

+		 email.based.monitor.address=jobs@sample.org

+		 email.based.monitor.password=SamplePassword

+	-  Zookeeper Server Configuration
+		- For 'Production' scenario make;

+		embedded.zk=false
+	- AMQP Notification Configuration
+		- Users can use RabbitMQ as 'Guest' users. This is the easy method. For this uncomment 

+		rabbitmq.broker.url=amqp://localhost:5672
+		- To use as a 'Production' user 

+		Navigate to RabbitMQ bin folder.

+		Make sure the RabbitMQ server is running. For production use 
rabbitmq-server -detached

+		Create a virtual-host and user with a password. Follow documentation in RabbitMQ Users & VirtualHost
+		To create a user; 
rabbitmqctl add_user airavata airavata

+		To create a vitrual-host 
rabbitmqctl add_vhost messaging

+		Provide permission to created username; 'airavata' to the created vhost 
rabbitmqctl set_permissions -p messaging airavata ".*" ".*" ".*”

+		Uncomment rabbitmq.broker.url=amqp://airavata:airavata@localhost:5672/messaging.
+11. Download and install Zookeeper. Use Download Zookeeper 
 You can download and install Zookeeper in the above created local folder; LocalAiravata
+12. Start Zookeeper
+    - Copy the sample config file to zoo.cfg:  cp conf/zoo_sample.cfg conf/zoo.cfg
+    - Navigate to the Zookeeper bin directory and start zookeeper 
zkServer.sh start

+13. In bin start the Airavata server and monitor log messages; This may require JAVA_HOME to be defined. Some configurations such as in  bin/zoo.cfg and bin/airavata-server.properties  may have to be adjusted if some ports are already in use. Ports need to be open as well.
+
sh airavata-server-start.sh all
 (This will run the airavata server in the background in demon mode)

+14. If you are in the target folder use given to start Airavata server;

+
sh apache-airavata-server-0.16-SNAPSHOT/bin/airavata-server-start.sh all

+15. To monitor the server starting up, view the airavata server log;

+
tail -f logs/airavata.log
	
+16. For subsequent Airavata copies; in the local Airavata folder where source code is cloned do a git clone https://github.com/apache/airavata.git for the latest trunk.
+
+Next is PGA Installation on CentOS 7
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/airavata-docs/blob/9259dfe4/0.16/Airavata-Upgrades.md
----------------------------------------------------------------------
diff --git a/0.16/Airavata-Upgrades.md b/0.16/Airavata-Upgrades.md
new file mode 100644
index 0000000..a06446e
--- /dev/null
+++ b/0.16/Airavata-Upgrades.md
@@ -0,0 +1,41 @@
+## Airavata Update
+
+1. In order to update Airavata with latest master code; go to the folder (Created in installation; LocalAiravata) initially created to clone Airavata.
+2. Within your local folder navigate to folder 'airavata' and execute 
git pull 

+Hint: If you are in the wrong folder you would probably get message 
'fatal: Not a git repository (or any of the parent directories): .git'

+3. Now build the code 
+
 mvn clean install

+Hint: To avoid the 'tests on install' run 
mvn clean install -Dmaven.test.skip=true

+4. Once the build is success, stop the running Airavata server. You can either stop the java server using 
+
kill -9 <process ID>

+OR

+Navigate to bin folder where airavata server exists (LocalAiravata/apache-airavata-server-0.15-SNAPSHOT/bin). and stop the server
+
./airavata-server.sh stop -force

+5. For references, back up the currently existing release
+
mv apache-airavata-server-0.15-SNAPSHOT apache-airavata-server-0.15-SNAPSHOT-bk

+6. Copy the new release to your local folder (LocalAiravata)
+
cp /<path to created folder>/LocalAiravata/airavata/distribution/target/apache-airavata-server-0.16-SNAPSHOT-bin.tar.gz .

+7. Un-tar the copied new release
+
tar -xvf apache-airavata-server-0.16-SNAPSHOT-bin.tar.gz

+8. Navigate to the new bin folder and back up airavata-server.properties and gfac-config.yaml files.
+9. Copy your previously used airavata-server.properties from the backed up release to bin folder (This is the easiest way to get the properties file updated. If you prefer, you can change the new file manually. ☺)
+
/<your local path>/LocalAiravata/LocalAiravata/apache-airavata-server-0.16-SNAPSHOT-bk/bin/airavata-server.properties .

+10. Copy your previously used gfac-config.yaml from the backed up release
+
cp /<your local path>/LocalAiravata/apache-airavata-server-0.16-SNAPSHOT-bk/bin/gfac-config.yaml .

+11. Now compare with the new airavata-server.properties and gfac-config.yaml and make necessary changes in the copied files.
+Try 
 diff airavata-server.properties airavata-server.properties-bk

+12. Check the path correctness of credential store keystore in airavata-server.properties file.
+
credential.store.keystore.url=/<your local path>/LocalAiravata/airavata_sym.jks

+13. Go to lib folder and copy mySQL jar (mysql-connector-java-5.1.38-bin.jar) from old backed up lib to the new lib
+
  cp /home/airavata/LocalAiravata/apache-airavata-server-0.16-SNAPSHOT-bk/lib/mysql-connector-java-5.1.38-bin.jar .

+14. Now restart airavata server in bin folder
+
./airavata-server.sh start

+15. Airavata ready for job submissions!
+16. For PGA updating steps try PGA Update
+
+
+
+
+ 
+
+

http://git-wip-us.apache.org/repos/asf/airavata-docs/blob/9259dfe4/0.16/AiravataApi.md
----------------------------------------------------------------------
diff --git a/0.16/AiravataApi.md b/0.16/AiravataApi.md
new file mode 100644
index 0000000..db615a5
--- /dev/null
+++ b/0.16/AiravataApi.md
@@ -0,0 +1,77 @@
+## Apache Airavata API
+
+For Airavata API documentation please visit Airavata 0.16 API Documentation
+### 
Airavata APIs for Experiments & Projects

+
+|       Gateway Function/Feature        |           Airavata API            |           Description         |
+|:--------------------------------------|:----------------------------------|:------------------------------|
+Project
+| Create a Project                          | createProject                                      | Linked with Create Project in PGA.    |
+| Update Project                            | updateProject                                      | To update Project name and description.   |
+| Get a Project                             | getProject                                            | Retrieve Project by providing the ID.           |
+| Search Project by Name                    | searchProjectsByProjectName          | Search for Project by giving part or full project name.          |
+| Search Project by Desc                    | searchProjectsByProjectDesc          | Search for Project by giving part or full project description.          |
+| Get all user Projects                     | getUserProjects                                  | Retrieve all Projects of a user.          |
+Experiment
+| Create an Experiment                      | createExperiment                                | Create an Experiment.          |
+| Update an Experiment                      | updateExperiment                                | Update ab Experiment. Experiments with CREATED exp-status can be updated.          |
+| Get an Experiment                         | getExperiment                                      | Retrieve Experiment by providing the experiment ID.          |
+| Get Detailed Experiment                   | getDetailedExperimentTree              | Retrieve detailed Experiment by providing the experiment ID.          |
+| Clone an Experiment                       | cloneExperiment                                  | Clone an existing Experiment. Experiment with any exp-status can be cloned by providing the ID.          |
+| Cancel an Experiment                      | terminateExperiment                          | Cancel an existing Experiment. Experiments with exp-statuses LAUNCHING or EXECUTING can be cancelled.           |
+| Search Experiment by Name                 | searchExperimentsByName                  | Search Experiment by giving full or part of the name.          |
+| Search Experiment by Desc                 | searchExperimentsByDesc                  | Search Experiment by giving full or part of the description.           |
+| Search Experiments by Application Name    | searchExperimentsByApplication    | Search Experiment by giving the application name.           |
+| Search Experiment by Creation Time        | searchExperimentsByCreationTime  | Search Experiment by giving creationTime period.           |
+| Get all user Experiments                  | getUserExperiments                            | Search for all the experiments of a single user.          |
+| Get Experiments for a Project             | getExperimentsInProject                  | Retrieve all the experiments in a particular Project.          |
+
+
+
+
+
+### 
Airavata APIs for Admin Dashboard

+   
+|           Admin Function/Feature              |              Airavata API             |                                          Description                                              |
+|:----------------------------------------------|:--------------------------------------|:------------------------------------------------------------------------------------------------  |
+Add Gateway
+| Add a gateway                                 | addGateway        | Adding a new Gateway. |
+Credential Store  
+| Generate a Token/SSH Key                      | generateAndRegisterSSHKeys            | Generate new SSH Key and Token.   |
+| Get all Credential Store Tokens               | getAllGatewaySSHPubKeys               | Retrieve all the generated keys of a Gateway.              |
+| Remove a Token/SSH Key                        | deleteSSHPubKey                       | Select and delete a particular SSH Key Token pair.   |  
+Compute Resource (CR)                                                                                            
+| Get all application deployed CRs              | getAllApplicationDeployments          | Get all Application deployed Compute Resources.        |
+| Get a CR                                      | getComputeResource                    | Retrieve Compute Resource information by providing the resource ID.   |
+| Register CR                                   | registerComputeResource               | Register a new Compute Resource. This is Super Admin Feature.|
+| Update CR                                     | updateComputeResource                 | Retrieve an existing Compute Resource and update.|
+| Enable and Disable CR                         | getComputeResource    updateComputeResource    | Retrieve the CR and enable or disable through update. This is a Super Admin feature.| 
+| Delete a Queue                                | deleteBatchQueue                      | Delete a selected Queue from the Compute Resource.      |
+Storage Resource (SR) 
+| Get all SR Names                              | getAllStorageResourceNames          | Retrieve all storage resources of the gateway.  |
+| Get a SR                                      | getStorageResource                    | Fetch Storage Resource by providing the Storage ID.   | 
+| Register a SR                                 | registerStorageResource               | Register a new Storage Resource. This is a Super Admin Feature.|
+| Update a SR                                   | updateStorageResource                 | Update and existing Storage Resource.|
+| Enable and Disable SR                         | getStorageResource    updateStorageResource    |Retrieve the SR and enable or disable through update. This is a Super Admin feature.|
+| Delete SR                                     | deleteStorageResource                 |Select and delete and existing Storage Resource. |
+| Delete Data Movement Interface of SR          | deleteDataMovementInterface       |Delete a Data Movement Interface of a Storage Resource|
+Experiment Statistics
+| Get Experiment Statistics                     | getExperimentStatistics               | Displays experiments grouped by the experiment status and derived for the given date time range.  |
+Gateway Preferences
+| Add CR Preference for a gateway               |  addGatewayComputeResourcePreference     ||     
+| Edit CR Preference                            |  addGatewayComputeResourcePreference     ||
+| Delete a CR Preference                        |  ||
+| Add SR Preference for a gateway               |  addGatewayStoragePreference     || 
+| Edit SR Preference                            |  addGatewayStoragePreference     ||
+| Delete a SR Preference                        |      ||
+Other
+| Get all Notices                               | noticesView                           | View all existing Notices.        |
+
+
+
+
+
+


+If any questions or clarification regarding the API documentation please contact us through;
+
Airavata Mailing List 
 OR

+HipChat

http://git-wip-us.apache.org/repos/asf/airavata-docs/blob/9259dfe4/0.16/Application-CookBook.md
----------------------------------------------------------------------
diff --git a/0.16/Application-CookBook.md b/0.16/Application-CookBook.md
new file mode 100644
index 0000000..ae69e4e
--- /dev/null
+++ b/0.16/Application-CookBook.md
@@ -0,0 +1,43 @@
+## Applications to Run in Gateway
+
+### Few Applications Supported by Airavata
+This list contains a sub section of application supported via gateways linked to Airavata. 
+We can help you to run these in your own gateway.
+
+|  	Application  	|  		Availability  					|
+|:-----------------	|:-------------------------------------	|
+|  Abaqus			|	Comet								|
+|  Abaqus-6.13-3	|	Comet								|
+|  Abinit			|	BigRed, Bridges, Comet				|
+|  Amber_Sander		|	BigRed, Comet, Stampede				|
+|  AutoDock			|	BigRed, Karst						|
+|  AutoDock_Vina	|	Comet, Stampede						|
+|  CP2K				|	Comet, Stampede						|
+|  CPMD				|	Comet								|
+|  DDSCat			|	Comet								|
+|  DFTB+			|	Stampede							|
+|  Gamess			|	BigRed, Comet, Gordon, Stampede		|
+|  Gaussian			|	BigRed. Bridges, Comet, Gordon		|
+|  Gaussian16		|	Bridges								|
+|  Gromacs			|	BigRed, Bridges, Comet, Stampede	|
+|  Lammps			|	BigRed, Comet, Gordon, Stampede		|
+|  Molcas			|	Stampede KNL, Stampede				|
+|  NEK5000			|	BigRed								|
+|  NWChem			|	Comet, Stampede						|
+|  Phasta_P			|	Stampede							|
+|  PSI4.0			|	Comet								|
+|  QChem			|	Comet								|
+|  Quantum_Espresso	|	Comet, Bridges, JetStream, Stampede	|
+|  Tinker_Monte		|	Stampede							|
+|  Trinity			|	Karst								|
+|  WRF				|	Stampede							|
+
+
+
+### Application Inputs
+Inputs for all applications currently available in Airavata can be found Here!
+
+
+
+### Application Configuration
+For sample configurations please refer Admin Tutorials
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/airavata-docs/blob/9259dfe4/0.16/CPI.md
----------------------------------------------------------------------
diff --git a/0.16/CPI.md b/0.16/CPI.md
new file mode 100644
index 0000000..a1adde3
--- /dev/null
+++ b/0.16/CPI.md
@@ -0,0 +1,291 @@
+
+

+
Thrift module: sharing_cpi

+
+
+
+
+
+
ModuleServicesData typesConstants
sharing_cpiSharingRegistryService

+
+

+
Services

+
Service: SharingRegistryService

+
Function: SharingRegistryService.createDomain

+
string createDomain(sharing_models.Domain domain)
+    throws sharing_models.SharingRegistryException
+
API method to create a new domain

+
Function: SharingRegistryService.updateDomain

+
bool updateDomain(sharing_models.Domain domain)
+    throws sharing_models.SharingRegistryException
+
API method to update a domain

+
Function: SharingRegistryService.deleteDomain

+
bool deleteDomain(string domainId)
+    throws sharing_models.SharingRegistryException
+
API method to delete domain

+
Function: SharingRegistryService.getDomain

+
sharing_models.Domain getDomain(string domainId)
+    throws sharing_models.SharingRegistryException
+
API method to retrieve a domain

+
Function: SharingRegistryService.getDomains

+
list<sharing_models.Domain> getDomains(i32 offset,
+                                       i32 limit)
+    throws sharing_models.SharingRegistryException
+
API method to get all domain.

+
Function: SharingRegistryService.createUser

+
string createUser(sharing_models.User user)
+    throws sharing_models.SharingRegistryException
+
API method to register a user in the system

+
Function: SharingRegistryService.updatedUser

+
bool updatedUser(sharing_models.User user)
+    throws sharing_models.SharingRegistryException
+
API method to update existing user

+
Function: SharingRegistryService.deleteUser

+
bool deleteUser(string domainId,
+                string userId)
+    throws sharing_models.SharingRegistryException
+
API method to delete user

+
Function: SharingRegistryService.getUser

+
sharing_models.User getUser(string domainId,
+                            string userId)
+    throws sharing_models.SharingRegistryException
+
API method to get a user

+
Function: SharingRegistryService.getUsers

+
list<sharing_models.User> getUsers(string domainId,
+                                   i32 offset,
+                                   i32 limit)
+    throws sharing_models.SharingRegistryException
+
API method to get a list of users in a specific domain.

+
  • domainId : Domain id
    • 
+
  • offset : Starting result number
    • 
+
  • limit : Number of max results to be sent
    • 
+
    Function: SharingRegistryService.createGroup
    
+
    string createGroup(sharing_models.UserGroup group)
+    throws sharing_models.SharingRegistryException
+
    API method to create a new group
    
+
    Function: SharingRegistryService.updateGroup
    
+
    bool updateGroup(sharing_models.UserGroup group)
+    throws sharing_models.SharingRegistryException
+
    API method to update a group
    
+
    Function: SharingRegistryService.deleteGroup
    
+
    bool deleteGroup(string domainId,
+                 string groupId)
+    throws sharing_models.SharingRegistryException
+
    API method to delete a group
    
+
    Function: SharingRegistryService.getGroup
    
+
    sharing_models.UserGroup getGroup(string domainId,
+                                  string groupId)
+    throws sharing_models.SharingRegistryException
+
    API method to get a group
    
+
    Function: SharingRegistryService.getGroups
    
+
    list<sharing_models.UserGroup> getGroups(string domainId,
+                                         i32 offset,
+                                         i32 limit)
+
    API method to get groups in a domainId.
    
+
    Function: SharingRegistryService.addUsersToGroup
    
+
    bool addUsersToGroup(string domainId,
+                     list<string> userIds,
+                     string groupId)
+    throws sharing_models.SharingRegistryException
+
    API method to add list of users to a group
    
+
    Function: SharingRegistryService.removeUsersFromGroup
    
+
    bool removeUsersFromGroup(string domainId,
+                          list<string> userIds,
+                          string groupId)
+    throws sharing_models.SharingRegistryException
+
    API method to remove users from a group
    
+
    Function: SharingRegistryService.getGroupMembersOfTypeUser
    
+
    list<sharing_models.User> getGroupMembersOfTypeUser(string domainId,
+                                                    string groupId,
+                                                    i32 offset,
+                                                    i32 limit)
+    throws sharing_models.SharingRegistryException
+
    API method to get list of child users in a group. Only the direct members will be returned.
    
+
    Function: SharingRegistryService.getGroupMembersOfTypeGroup
    
+
    list<sharing_models.UserGroup> getGroupMembersOfTypeGroup(string domainId,
+                                                          string groupId,
+                                                          i32 offset,
+                                                          i32 limit)
+    throws sharing_models.SharingRegistryException
+
    API method to get list of child groups in a group. Only the direct members will be returned.
    
+
    Function: SharingRegistryService.addChildGroupsToParentGroup
    
+
    bool addChildGroupsToParentGroup(string domainId,
+                                 list<string> childIds,
+                                 string groupId)
+    throws sharing_models.SharingRegistryException
+
    API method to add a child group to a parent group.
    
+
    Function: SharingRegistryService.removeChildGroupFromParentGroup
    
+
    bool removeChildGroupFromParentGroup(string domainId,
+                                     string childId,
+                                     string groupId)
+    throws sharing_models.SharingRegistryException
+
    API method to remove a child group from parent group.
    
+
    Function: SharingRegistryService.createEntityType
    
+
    string createEntityType(sharing_models.EntityType entityType)
+    throws sharing_models.SharingRegistryException
+
    API method to create a new entity type
    
+
    Function: SharingRegistryService.updateEntityType
    
+
    bool updateEntityType(sharing_models.EntityType entityType)
+    throws sharing_models.SharingRegistryException
+
    API method to update entity type
    
+
    Function: SharingRegistryService.deleteEntityType
    
+
    bool deleteEntityType(string domainId,
+                      string entityTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to delete entity type
    
+
    Function: SharingRegistryService.getEntityType
    
+
    sharing_models.EntityType getEntityType(string domainId,
+                                        string entityTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to get an entity type
    
+
    Function: SharingRegistryService.getEntityTypes
    
+
    list<sharing_models.EntityType> getEntityTypes(string domainId,
+                                               i32 offset,
+                                               i32 limit)
+    throws sharing_models.SharingRegistryException
+
    API method to get entity types in a domainId.
    
+
    Function: SharingRegistryService.createEntity
    
+
    string createEntity(sharing_models.Entity entity)
+    throws sharing_models.SharingRegistryException
+
    API method to register new entity
    
+
    Function: SharingRegistryService.updateEntity
    
+
    bool updateEntity(sharing_models.Entity entity)
+    throws sharing_models.SharingRegistryException
+
    API method to update entity
    
+
    Function: SharingRegistryService.deleteEntity
    
+
    bool deleteEntity(string domainId,
+                  string entityId)
+    throws sharing_models.SharingRegistryException
+
    API method to delete entity
    
+
    Function: SharingRegistryService.getEntity
    
+
    sharing_models.Entity getEntity(string domainId,
+                                string entityId)
+    throws sharing_models.SharingRegistryException
+
    API method to get entity
    
+
    Function: SharingRegistryService.searchEntities
    
+
    list<sharing_models.Entity> searchEntities(string domainId,
+                                           string userId,
+                                           list<sharing_models.SearchCriteria> filters,
+                                           i32 offset,
+                                           i32 limit)
+    throws sharing_models.SharingRegistryException
+
    API method to search entities
    
+
    Function: SharingRegistryService.getListOfSharedUsers
    
+
    list<sharing_models.User> getListOfSharedUsers(string domainId,
+                                               string entityId,
+                                               string permissionTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to get a list of shared users given the entity id
    
+
    Function: SharingRegistryService.getListOfSharedGroups
    
+
    list<sharing_models.UserGroup> getListOfSharedGroups(string domainId,
+                                                     string entityId,
+                                                     string permissionTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to get a list of shared groups given the entity id
    
+
    Function: SharingRegistryService.createPermissionType
    
+
    string createPermissionType(sharing_models.PermissionType permissionType)
+    throws sharing_models.SharingRegistryException
+
    API method to create permission type
    
+
    Function: SharingRegistryService.updatePermissionType
    
+
    bool updatePermissionType(sharing_models.PermissionType permissionType)
+    throws sharing_models.SharingRegistryException
+
    API method to update permission type
    
+
    Function: SharingRegistryService.deletePermissionType
    
+
    bool deletePermissionType(string domainId,
+                          string permissionTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to delete permission type
    
+
    Function: SharingRegistryService.getPermissionType
    
+
    sharing_models.PermissionType getPermissionType(string domainId,
+                                                string permissionTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to get permission type
    
+
    Function: SharingRegistryService.getPermissionTypes
    
+
    list<sharing_models.PermissionType> getPermissionTypes(string domainId,
+                                                       i32 offset,
+                                                       i32 limit)
+    throws sharing_models.SharingRegistryException
+
    API method to get list of permission types in a given domainId.
    
+
    Function: SharingRegistryService.shareEntityWithUsers
    
+
    bool shareEntityWithUsers(string domainId,
+                          string entityId,
+                          list<string> userList,
+                          string perssionTypeId,
+                          bool cascadePermission)
+    throws sharing_models.SharingRegistryException
+
    API method to share an entity with users
    
+
    Function: SharingRegistryService.revokeEntitySharingFromUsers
    
+
    bool revokeEntitySharingFromUsers(string domainId,
+                                  string entityId,
+                                  list<string> userList,
+                                  string perssionTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to revoke sharing from a list of users
    
+
    Function: SharingRegistryService.shareEntityWithGroups
    
+
    bool shareEntityWithGroups(string domainId,
+                           string entityId,
+                           list<string> groupList,
+                           string perssionTypeId,
+                           bool cascadePermission)
+    throws sharing_models.SharingRegistryException
+
    API method to share an entity with list of groups
    
+
    Function: SharingRegistryService.revokeEntitySharingFromGroups
    
+
    bool revokeEntitySharingFromGroups(string domainId,
+                                   string entityId,
+                                   list<string> groupList,
+                                   string perssionTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to revoke sharing from list of users
    
+
    Function: SharingRegistryService.userHasAccess
    
+
    bool userHasAccess(string domainId,
+                   string userId,
+                   string entityId,
+                   string permissionTypeId)
+    throws sharing_models.SharingRegistryException
+
    API method to check whether a user has access to a specific entity
    
+
    
+

http://git-wip-us.apache.org/repos/asf/airavata-docs/blob/9259dfe4/0.16/Contact-Us.md
----------------------------------------------------------------------
diff --git a/0.16/Contact-Us.md b/0.16/Contact-Us.md
new file mode 100644
index 0000000..65f6966
--- /dev/null
+++ b/0.16/Contact-Us.md
@@ -0,0 +1,14 @@
+## Contact Us
+For your clarifications, Questions and Suggestions please contact us through
+Airavata mailing list
+
+OR
+
+HipChat
+
+Issue Reporting
+Any issue encountered with Airavata or our gateway please log through;
+
+Airavata JIRA
+

    
+NOTE: If you don't have an account in Apache JIRA you have to create one prior to submitting issues.