Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 77982200C30 for ; Tue, 7 Mar 2017 13:02:31 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 76154160B74; Tue, 7 Mar 2017 12:02:31 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 28042160B68 for ; Tue, 7 Mar 2017 13:02:30 +0100 (CET) Received: (qmail 65452 invoked by uid 500); 7 Mar 2017 12:02:29 -0000 Mailing-List: contact commits-help@zeppelin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zeppelin.apache.org Delivered-To: mailing list commits@zeppelin.apache.org Received: (qmail 65443 invoked by uid 99); 7 Mar 2017 12:02:29 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 Mar 2017 12:02:29 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 3C5C3DFBD3; Tue, 7 Mar 2017 12:02:29 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: ahyoungryu@apache.org To: commits@zeppelin.apache.org Message-Id: <352ad992281d4509818d98131be1c5b3@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: zeppelin git commit: [ZEPPELIN-2109][2110] Sortable Helium pkgs & Introduce "INTERPRETER" type Date: Tue, 7 Mar 2017 12:02:29 +0000 (UTC) archived-at: Tue, 07 Mar 2017 12:02:31 -0000 Repository: zeppelin Updated Branches: refs/heads/master 79ace932a -> 142597bcf [ZEPPELIN-2109][2110] Sortable Helium pkgs & Introduce "INTERPRETER" type ### What is this PR for? * Make Helium pkgs sortable For now, we have 3 types of Helium packages: `VISUALIZATION`, `SPELL` and `APPLICATION`. `VISUALIZATION` and `SPELL` type of pkgs can be published [NPM registry](https://www.npmjs.com/). Likewise, `APPLICATION` type pkg can be registered in [Maven central repository](http://search.maven.org/). But all available Helium packages are not sorted and shown in random order in Helium menu. To do this, I put "NPM Packages" & "Maven Artifacts" button at top-right corner of Helium menu. * Introduce "INTERPRETER" type As a first step of [ZEPPELIN-1993: Install interpreter from Helium menu](https://issues.apache.org/jira/browse/ZEPPELIN-1993), I added `INTERPRETER` type as a new Helium pkg type. Like [ZEPPELIN-1973](https://issues.apache.org/jira/browse/ZEPPELIN-1973) did, we can retrieve Maven artifact(which has `zeppelin-interpreter` as its dependency) info and save it to Helium online registry as well. For more detailed explanation, please see [ZEPPELIN-1993](https://issues.apache.org/jira/browse/ZEPPELIN-1993) and [ZEPPELIN-2110](https://issues.apache.org/jira/browse/ZEPPELIN-2110)'s description. ### What type of PR is it? Improvement ### What is the Jira issue? * [ZEPPELIN-2109](https://issues.apache.org/jira/browse/ZEPPELIN-2109) : Make Helium packages sortable in Helium menu * [ZEPPELIN-2110](https://issues.apache.org/jira/browse/ZEPPELIN-2110) : List community & 3rd party interpreter registered at Maven central repo in Helium menu ### How should this be tested? To see how `INTERPRETER` type can be shown in Helium menu, 1. Replace temporarily [ZeppelinConfiguration.java#L46](https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java#L46) with [this url](https://raw.githubusercontent.com/AhyoungRyu/spark-notebook-example/master/helium-test.json) 2. Build \w below command and restart ``` $ mvn clean package -DskipTests -pl 'zeppelin-zengine, zeppelin-server, zeppelin-interpreter' ``` 3. Start web dev server under `ZEPPELIN_HOME/zeppelin-web` and browse `localhost:9000` ``` $ yarn run dev:helium ``` 4. Go to Helium menu ### Screenshots (if appropriate) - How's it look? ![record](https://cloud.githubusercontent.com/assets/10060731/23219279/02fc234e-f962-11e6-86a9-30e7dc0953aa.gif) - Package selection by types (`VISUALIZATION`, `SPELL`, `INTERPRETER` and `APPLICATION`) screen shot 2017-02-23 at 12 51 22 am - Disabled "Enable" button until [ZEPPELIN-1993](https://issues.apache.org/jira/browse/ZEPPELIN-1993) is resolved screen shot 2017-02-20 at 2 03 23 am ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: AhyoungRyu Author: Ahyoung Ryu Closes #2037 from AhyoungRyu/ZEPPELIN-2109 and squashes the following commits: 7393c6a [AhyoungRyu] Change '==' to '===' to make CI green 0fd68ba [AhyoungRyu] Remove useless comment code 569f9c9 [AhyoungRyu] Add guide dialog msg for INTERPRETER type pkg e6ccc28 [AhyoungRyu] Fix typo in docs bb34428 [AhyoungRyu] Package orderBy publishedDate a5eb006 [AhyoungRyu] Add 'spell' to description 1707882 [AhyoungRyu] Get publishedDate, artifactId and groupId from online registry data 03312ac [AhyoungRyu] Fix interpreter type name duplication 95b0b87 [AhyoungRyu] Fix mistakes made by resolving conflicts 894a27e [Ahyoung Ryu] Merge branch 'master' into ZEPPELIN-2109 d5ae4e2 [AhyoungRyu] Add default msg to show when each package size is 0 899e8fa [AhyoungRyu] Put license sentences back removed by mistake c68efd2 [AhyoungRyu] Remove unused img files 6d7f4fa [AhyoungRyu] Make packages sortable by type not repository 86e1eda [AhyoungRyu] Remove unnecassary words b74c599 [AhyoungRyu] Remove console.log 4a6eace [AhyoungRyu] Enable Helium pkg type sortable f1c7bac [AhyoungRyu] Add 'INTERPRETER', 'APPLICATION' as helium pkg types 6de9169 [AhyoungRyu] Add img files Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/142597bc Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/142597bc Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/142597bc Branch: refs/heads/master Commit: 142597bcf0558ede33f64d265f21730127ff496b Parents: 79ace93 Author: AhyoungRyu Authored: Tue Mar 7 11:36:28 2017 +0900 Committer: ahyoungryu Committed: Tue Mar 7 21:02:20 2017 +0900 ---------------------------------------------------------------------- docs/manual/interpreterinstallation.md | 4 +- .../apache/zeppelin/helium/HeliumPackage.java | 16 +++ .../src/app/helium/helium.controller.js | 133 ++++++++++++++----- zeppelin-web/src/app/helium/helium.css | 31 ++++- zeppelin-web/src/app/helium/helium.html | 70 ++++++++-- .../src/assets/images/maven_default_icon.png | Bin 0 -> 4248 bytes .../src/components/helium/helium-type.js | 2 + .../zeppelin/helium/HeliumBundleFactory.java | 2 +- 8 files changed, 207 insertions(+), 51 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/142597bc/docs/manual/interpreterinstallation.md ---------------------------------------------------------------------- diff --git a/docs/manual/interpreterinstallation.md b/docs/manual/interpreterinstallation.md index 57bd6af..5825d1d 100644 --- a/docs/manual/interpreterinstallation.md +++ b/docs/manual/interpreterinstallation.md @@ -111,9 +111,9 @@ You can also install 3rd party interpreters located in the maven repository by u ./bin/install-interpreter.sh --name interpreter1 --artifact groupId1:artifact1:version1 ``` -The above command will download maven artifact `groupId1:artifact1:version1` and all of it's transitive dependencies into `interpreter/interpreter1` directory. +The above command will download maven artifact `groupId1:artifact1:version1` and all of its transitive dependencies into `interpreter/interpreter1` directory. -After restart Zeppelin, then [create interpreter setting](../manual/interpreters.html#what-is-zeppelin-interpreter) and [bind it with your notebook](../manual/interpreters.html#what-is-zeppelin-interpreter-setting). +After restart Zeppelin, then [create interpreter setting](../manual/interpreters.html#what-is-zeppelin-interpreter) and [bind it with your note](../manual/interpreters.html#what-is-zeppelin-interpreter-setting). #### Install multiple 3rd party interpreters at once http://git-wip-us.apache.org/repos/asf/zeppelin/blob/142597bc/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java index 62c4bcf..c1af75f 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java @@ -34,6 +34,10 @@ public class HeliumPackage { // [[ .. and .. and .. ] or [ .. and .. and ..] ..] private String license; private String icon; + private String published; + + private String groupId; // get groupId of INTERPRETER type package + private String artifactId; // get artifactId of INTERPRETER type package private SpellPackageInfo spell; private Map config; @@ -108,6 +112,18 @@ public class HeliumPackage { return icon; } + public String getPublishedDate() { + return published; + } + + public String getGroupId() { + return groupId; + } + + public String getArtifactId() { + return artifactId; + } + public SpellPackageInfo getSpellInfo() { return spell; } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/142597bc/zeppelin-web/src/app/helium/helium.controller.js ---------------------------------------------------------------------- diff --git a/zeppelin-web/src/app/helium/helium.controller.js b/zeppelin-web/src/app/helium/helium.controller.js index 5a19ea0..3819133 100644 --- a/zeppelin-web/src/app/helium/helium.controller.js +++ b/zeppelin-web/src/app/helium/helium.controller.js @@ -23,7 +23,15 @@ export default function HeliumCtrl($scope, $rootScope, $sce, $scope.showVersions = {}; $scope.bundleOrder = []; $scope.bundleOrderChanged = false; + $scope.vizTypePkg = {} + $scope.spellTypePkg = {} + $scope.intpTypePkg = {} + $scope.appTypePkg = {} + $scope.numberOfEachPackageByType = {} + $scope.allPackageTypes = [HeliumType][0] + $scope.pkgListByType = 'VISUALIZATION' $scope.defaultPackageConfigs = {}; // { pkgName, [{name, type, desc, value, defaultValue}] } + $scope.intpDefaultIcon = $sce.trustAsHtml(''); function init() { // get all package info and set config @@ -31,6 +39,7 @@ export default function HeliumCtrl($scope, $rootScope, $sce, .then(({ pkgSearchResults, defaultPackages }) => { $scope.pkgSearchResults = pkgSearchResults; $scope.defaultPackages = defaultPackages; + classifyPkgType($scope.defaultPackages) return heliumService.getAllPackageConfigs() }) .then(defaultPackageConfigs => { @@ -45,6 +54,38 @@ export default function HeliumCtrl($scope, $rootScope, $sce, }); } + var classifyPkgType = function(packageInfos) { + var vizTypePkg = {} + var spellTypePkg = {} + var intpTypePkg = {} + var appTypePkg = {} + + for (var name in packageInfos) { + var pkgs = packageInfos[name] + var pkgType = pkgs.pkg.type + + switch (pkgType) { + case HeliumType.VISUALIZATION: + vizTypePkg[name] = pkgs; + break; + case HeliumType.SPELL: + spellTypePkg[name] = pkgs; + break; + case HeliumType.INTERPRETER: + intpTypePkg[name] = pkgs; + break; + case HeliumType.APPLICATION: + appTypePkg[name] = pkgs; + break; + } + } + + $scope.vizTypePkg = vizTypePkg + $scope.spellTypePkg = spellTypePkg + $scope.appTypePkg = appTypePkg + $scope.intpTypePkg = intpTypePkg + }; + $scope.bundleOrderListeners = { accept: function(sourceItemHandleScope, destSortableScope) {return true;}, itemMoved: function(event) {}, @@ -108,40 +149,58 @@ export default function HeliumCtrl($scope, $rootScope, $sce, return license; } - $scope.enable = function(name, artifact) { + $scope.enable = function(name, artifact, type, groupId) { var license = getLicense(name, artifact); - - var confirm = BootstrapDialog.confirm({ - closable: false, - closeByBackdrop: false, - closeByKeyboard: false, + var mavenArtifactInfoToHTML = groupId +':'+ artifact.split('@')[0] + ':' + artifact.split('@')[1]; + var zeppelinVersion = $rootScope.zeppelinVersion; + var url = 'https://zeppelin.apache.org/docs/' + zeppelinVersion + '/manual/interpreterinstallation.html'; + + var confirm = '' + if (type === 'INTERPRETER') { + confirm = BootstrapDialog.show({ title: '', - message: 'Do you want to enable ' + name + '?' + - '
' + artifact + '
' + - '
License
' + - '
' + license + '
', - callback: function(result) { - if (result) { - confirm.$modalFooter.find('button').addClass('disabled'); - confirm.$modalFooter.find('button:contains("OK")') - .html(' Enabling'); - heliumService.enable(name, artifact). - success(function(data, status) { - init(); - confirm.close(); - }). - error(function(data, status) { - confirm.close(); - console.log('Failed to enable package %o %o. %o', name, artifact, data); - BootstrapDialog.show({ - title: 'Error on enabling ' + name, - message: data.message + message: '

Below command will download maven artifact ' + + '' + + mavenArtifactInfoToHTML + '' + + ' and all of its transitive dependencies into interpreter/interpreter-name directory.

' + + '

' +
+      './bin/install-interpreter.sh --name "interpreter-name" --artifact ' +
+        mavenArtifactInfoToHTML +'
' + + '

After restart Zeppelin, create interpreter setting and bind it with your note. ' + + 'For more detailed information, see Interpreter Installation.

' + }); + } else { + confirm = BootstrapDialog.confirm({ + closable: false, + closeByBackdrop: false, + closeByKeyboard: false, + title: '', + message: 'Do you want to enable ' + name + '?' + + '
' + artifact + '
' + + '
License
' + + '
' + license + '
', + callback: function (result) { + if (result) { + confirm.$modalFooter.find('button').addClass('disabled'); + confirm.$modalFooter.find('button:contains("OK")') + .html(' Enabling'); + heliumService.enable(name, artifact, type).success(function (data, status) { + init(); + confirm.close(); + }).error(function (data, status) { + confirm.close(); + console.log('Failed to enable package %o %o. %o', name, artifact, data); + BootstrapDialog.show({ + title: 'Error on enabling ' + name, + message: data.message + }); }); - }); - return false; + return false; + } } - } - }); + }); + } }; $scope.disable = function(name) { @@ -194,6 +253,20 @@ export default function HeliumCtrl($scope, $rootScope, $sce, !$scope.isLocalPackage(pkgSearchResult); }; + $scope.hasMavenLink = function(pkgSearchResult) { + const pkg = pkgSearchResult.pkg; + return (pkg.type === HeliumType.APPLICATION || pkg.type === HeliumType.INTERPRETER) && + !$scope.isLocalPackage(pkgSearchResult); + }; + + $scope.getPackageSize = function(pkgSearchResult, targetPkgType) { + var result = [] + _.map(pkgSearchResult, function (pkg) { + result.push(_.find(pkg, {type: targetPkgType})) + }) + return _.compact(result).length + } + $scope.configExists = function(pkgSearchResult) { // helium package config is persisted per version return pkgSearchResult.pkg.config && pkgSearchResult.pkg.artifact; http://git-wip-us.apache.org/repos/asf/zeppelin/blob/142597bc/zeppelin-web/src/app/helium/helium.css ---------------------------------------------------------------------- diff --git a/zeppelin-web/src/app/helium/helium.css b/zeppelin-web/src/app/helium/helium.css index 23e8c19..63a81ff 100644 --- a/zeppelin-web/src/app/helium/helium.css +++ b/zeppelin-web/src/app/helium/helium.css @@ -136,6 +136,33 @@ color: #636363; } +.heliumLearnMore { + margin-top:10px; +} + +.heliumLearnMore a { + cursor:pointer; + margin-right:10px; + text-decoration:none; +} + +.heliumRepoBtn { + margin-right: 8px; +} + +.heliumRepoBtn:hover, .heliumRepoBtn:focus { + margin-right: 8px; + outline: 0; +} + +.localPkgInfo { + margin: 10px 12px 0 0; + font-size: 11px; + font-style: italic; + color: #aaaaaa; + text-align: right; +} + .heliumConfig { margin-top: 30px; margin-bottom: 10px; @@ -147,10 +174,6 @@ margin-bottom: 15px; } -.heliumConfigValueInput { - -} - .heliumConfigValueText { vertical-align: top; } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/142597bc/zeppelin-web/src/app/helium/helium.html ---------------------------------------------------------------------- diff --git a/zeppelin-web/src/app/helium/helium.html b/zeppelin-web/src/app/helium/helium.html index 7718666..b64f71c 100644 --- a/zeppelin-web/src/app/helium/helium.html +++ b/zeppelin-web/src/app/helium/helium.html @@ -18,6 +18,23 @@ limitations under the License.

Helium

+
+ + + + +

* Local registry package's name is gray colored.

+
+ ng-show="bundleOrderChanged" + ng-click="saveBundleOrder()"> save @@ -44,28 +61,46 @@ limitations under the License.
+
+
+ Currently there is no available package to be listed +
+
- + ng-repeat="pkgSearchResult in defaultPackages | toArray:false | orderBy: 'pkg.published':true" + ng-show="$parent.pkgListByType === pkgSearchResult.pkg.type">
+
- {{pkgName}} + {{pkgSearchResult.pkg.name}} + + + {{pkgSearchResult.pkg.name}} - - {{pkgName}} + + + {{pkgSearchResult.pkg.name}} + {{pkgSearchResult.pkg.type}}
Enable
Disable
{{pkgSearchResult.pkg.artifact}} - + versions
    + ng-show="showVersions[pkgSearchResult.pkg.name]">
  • + ng-repeat="pkgSearchResult in pkgSearchResults[pkgSearchResult.pkg.name]"> {{pkgSearchResult.pkg.artifact}} - - enable + + see more +
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/142597bc/zeppelin-web/src/assets/images/maven_default_icon.png ---------------------------------------------------------------------- diff --git a/zeppelin-web/src/assets/images/maven_default_icon.png b/zeppelin-web/src/assets/images/maven_default_icon.png new file mode 100644 index 0000000..56b3894 Binary files /dev/null and b/zeppelin-web/src/assets/images/maven_default_icon.png differ http://git-wip-us.apache.org/repos/asf/zeppelin/blob/142597bc/zeppelin-web/src/components/helium/helium-type.js ---------------------------------------------------------------------- diff --git a/zeppelin-web/src/components/helium/helium-type.js b/zeppelin-web/src/components/helium/helium-type.js index 0ef4eb6..27b34fa 100644 --- a/zeppelin-web/src/components/helium/helium-type.js +++ b/zeppelin-web/src/components/helium/helium-type.js @@ -15,4 +15,6 @@ export const HeliumType = { VISUALIZATION: 'VISUALIZATION', SPELL: 'SPELL', + INTERPRETER: 'INTERPRETER', + APPLICATION: 'APPLICATION', } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/142597bc/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumBundleFactory.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumBundleFactory.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumBundleFactory.java index 62f8e02..de03195 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumBundleFactory.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumBundleFactory.java @@ -34,7 +34,7 @@ import java.net.URL; import java.util.*; /** - * Load helium visualization + * Load helium visualization & spell */ public class HeliumBundleFactory { Logger logger = LoggerFactory.getLogger(HeliumBundleFactory.class);